Systems and methods for determining optimal remediation recommendations in penetration testing

ABSTRACT

Methods and systems for providing a recommendation for improving the security of a networked system against attackers. The recommendation may include a recommendation of a single sub-goal to be protected to achieve optimal improvement in security, or of multiple such sub-goals. If the recommendation includes multiple sub-goals, the sub-goals may be ordered such that the first sub-goal is more important to protect, provides a greater benefit by being protected, or is more cost effective to protect than subsequent sub-goals in the ordered list of sub-goals.

RELATED APPLICATIONS

The present application gains priority from U.S. Provisional PatentApplication 62/870,742 filed on Jul. 4, 2019, and entitled “Systems andMethods for Determining Optimal Remediation Recommendations inPenetration Testing”.

BACKGROUND OF THE INVENTION

There is currently a proliferation of organizational networked computingsystems. Every type of organization, be it a commercial company, auniversity, a bank, a government agency or a hospital, heavily relies onone or more networks interconnecting multiple computing nodes. Failuresof the networked computing system of an organization, or even of only aportion of it, might cause significant damage, up to completely shuttingdown all operations. Additionally, much of the data of the organization,if not all the data, exist somewhere on its networked computing system,including all confidential data comprising the “crown jewels” of theorganization, such as prices, details of customers, purchase orders,employees' salaries, technical formulas, etc. Loss of such data or leaksof such data to unauthorized external entities might be disastrous forthe organization.

Many organizational networks are connected to the Internet at leastthrough one network node, and consequently may be subject to attacks bycomputer hackers or by hostile adversaries. Quite often the newspapersreport incidents in which websites crashed, sensitive data was stolen,or service to customers was denied, where the failures were the resultsof hostile penetration into an organization's networked computingsystem.

Thus, many organizations invest a lot of efforts and costs in preventivemeans designed to protect their computing networks against potentialthreats. There are many defensive products offered in the marketclaiming to provide protection against one or more known modes ofattack, and many organizations arm themselves to the teeth with multipleproducts of this kind.

However, it is difficult to tell how effective such products really arein achieving their stated goals of blocking hostile attacks, andconsequently most CISOs (Computer Information Security Officers) willadmit (maybe only off the record), that they don't really know how wellthey can withstand an attack from a given adversary. The only way toreally know the strength and security of a system, is by trying toattack it as a real adversary would. This is known as red-teaming orpenetration testing (pen testing, in short), and is a very commonapproach that is even required by regulation in some developedcountries.

Penetration testing requires highly talented people to man the testingteam. Those people should be familiar with each and every publicly knownvulnerability and attacking method and should also have a very goodfamiliarity with networking techniques and multiple operating systemsimplementations. Such people are hard to find and therefore manyorganizations give up establishing their own penetration testing teamsand resort to hiring external expert consultants for carrying out thatrole (or completely give up penetration testing). However, externalconsultants are expensive and therefore are typically called in only forbrief periods separated by long intervals in which no penetrationtesting is carried out. This makes the penetration testing ineffective,as vulnerabilities caused by new attacks, that appear almost daily, arediscovered only months after becoming serious threats to theorganization.

Additionally, even rich organizations that can afford hiring talentedexperts for in-house penetration testing teams do not achieve goodprotection. Testing for vulnerabilities of a large network containingmany types of computers, operating systems, network routers and otherdevices is both a very complex and a very tedious process. The processis prone to human errors such as missing testing for certain threats ormisinterpreting the damages of certain attacks. Additionally, because aprocess of full testing against all threats is quite long, theorganization might again end with a too long discovery period after anew threat appears.

In view of the above difficulties, several vendors offer automatedpenetration testing systems. Such systems automatically discover andreport vulnerabilities of a networked system, potential damages thatmight be caused to the networked system, and potential trajectories ofattack that may be employed by an attacker.

A penetration testing process involves at least the following mainfunctions: (i) a reconnaissance function, (ii) an attack function, and(iii) a reporting function. The process may also include additionalfunctions, for example a cleanup function that restores the testednetworked system to its original state as it was before the test. In anautomated penetration testing system, at least one of the above threefunctions is at least partially automated, and typically two or three ofthem are at least partially automated.

A reconnaissance function is the function within a penetration testingsystem that handles the collection of data about the tested networkedsystem. The collected data may include internal data of networks nodes,data about network traffic within the tested networked system, businessintelligence data of the organization owning the tested networkedsystem, etc. The functionality of a prior art reconnaissance functioncan be implemented, for example, by software executing in a server thatis not one of the network nodes of the tested networked system, wherethe server probes the tested networked system for the purpose ofcollecting data about it.

An attack function is the function within a penetration testing systemthat handles the determination of whether security vulnerabilities existin the tested networked system based on data collected by thereconnaissance function. The functionality of a prior art attackfunction can be implemented, for example, by software executing in aserver that is not one of the nodes of the tested networked system,where the server attempts to attack the tested networked system for thepurpose of verifying that it can be compromised.

A reporting function is the function within a penetration testing systemthat handles the reporting of results of the penetration testing system.The functionality of a prior art reporting function may be implemented,for example, by software executing in the same server that executes thefunctionality of the attack function, where the server reports thefindings of the attack function to an administrator or a CISO of thetested networked system.

FIG. 1A (PRIOR ART) is a block diagram of code modules of a typicalpenetration testing system. FIG. 1B (PRIOR ART) is a related flow-chart.

In FIG. 1A, code for the reconnaissance function, for the attackfunction, and for the reporting function are respectively labelled as20, 30 and 40, and are each schematically illustrated as part of apenetration testing system code module (PTSCM) labelled as 10. The term‘code’ is intended broadly and may include any combination ofcomputer-executable code and computer-readable data which when readaffects the output of execution of the code. The computer-executablecode may be provided as any combination of human-readable code (e.g. ina scripting language such as Python), machine language code, assemblercode and byte code, or in any form known in the art. Furthermore, theexecutable code may include any stored data (e.g. structured data) suchas configuration files, XML files, and data residing in any type ofdatabase (e.g. a relational database, an object-database, etc.).

In one example and as shown in FIG. 1B, the reconnaissance function(performed in step S21 by execution of reconnaissance function code 20),the attack function (performed in step S31 by execution of attackfunction code 30) and the reporting function (performed in step S41 byexecution of reporting function code 40) are executed in strictlysequential order so that first the reconnaissance function is performedby executing code 20 thereof, then the attack function is performed byexecuting code 30 thereof, and finally the reporting function isperformed 40 by executing code thereof.

However, the skilled artisan will appreciate that this order is just oneexample, and is not a requirement. For example, the attack and thereporting functions may be performed in parallel or in an interleavedway, with the reporting function reporting first results obtained by theattack function, while the attack function is working on additionalresults.

Similarly, the reconnaissance and the attack functions may operate inparallel or in an interleaved way, with the attack function detecting avulnerability based on first data collected by the reconnaissancefunction, while the reconnaissance function is working on collectingadditional data.

FIG. 1A also illustrates code of an optional cleanup function which islabeled as 50. Also illustrated in FIG. 1B is step S51 of performing acleanup function—e.g. by cleanup function code 50 of FIG. 1A.

“A campaign of penetration testing” is a specific run of a specific testof a specific networked system by the penetration testing system.

A penetration-testing-campaign module may comprise at least part ofreconnaissance function code 20, attack function code 30, reportingfunction code 40 and optionally cleanup function code 50—for example, incombination with suitable hardware (e.g. one or more computing device(s)110 and one or more processor(s) 120 thereof, see FIG. 2) for executingthe code.

FIG. 2 illustrates a prior art computing device 110 which may have anyform-factor including but not limited to a laptop, a desktop, a mobilephone, a server, a tablet, or any other form factor. The computingdevice 110 in FIG. 2 includes (i) computer memory 160 which may storecode 180; (ii) one or more processors 120 (e.g. central-processing-unit(CPU)) for executing code 180; (iii) one or more human-interfacedevice(s) 140 (e.g. mouse, keyboard, touchscreen, gesture-detectingapparatus including a camera, etc.) or an interface (e.g. USB interface)to receive input from a human-interface device; (iv) a display device130 (e.g. computer screen) or an interface (e.g. HDMI interface, USBinterface) for exporting video to a display device and (v) a networkinterface 150 (e.g. a network card, or a wireless modem).

Memory 160 may include any combination of volatile (e.g. RAM) andnon-volatile (e.g. ROM, flash, disk-drive) memory. Code 180 may includeoperating-system code—e.g. Windows®, Linux®, Android®, Mac-OS®.

Computing device 110 may include a user-interface for receiving inputfrom a user (e.g. manual input, visual input, audio input, or input inany other form) and for visually displaying output. The user-interface(e.g. graphical user interface (GUI)) of computing device 110 may thusinclude the combination of HID device 140 or an interface thereof (i.e.in communication with an external HID device), display device 130 or aninterface thereof (i.e. in communication with an external displaydevice), and user-interface (UI) code stored in memory 160 and executedby one or more processor(s) 120. The user-interface may include one ormore GUI widgets such as labels, buttons (e.g. radio buttons or checkboxes), sliders, spinners, icons, windows, panels, text boxes, and thelike.

In one example, a penetration testing system is the combination of (i)code 10 (e.g. including reconnaissance function code 20, attack functioncode 30, reporting function code 40, and optionally cleaning functioncode 50); and (ii) one or more computing devices 110 which execute thecode 10. For example, a first computing device may execute a firstportion of code 10 and a second computing device (e.g. in networkedcommunication with the first computing device) may execute a secondportion of code 10.

Penetration testing systems may employ different types of architectures,each having its advantages and disadvantages. Examples are actual attackpenetration testing systems, simulated penetration testing systems andreconnaissance agent penetration testing systems. See the Definitionssection for more details about these types of penetration testingsystems.

The Problem to Solve

As described above, an automated penetration testing system carries outa penetration testing campaign in order to test how well the testednetworked system is protected against cyber-attacks. Each penetrationtesting campaign has an associated “goal of an attacker”. The “goal ofan attacker” of a campaign indicates what the attacker of the campaignis attempting to achieve by attacking the networked system being tested.In other words, what is the criterion according to which the attack willbe considered a success or a failure and/or to what extent was theattack successful. Examples of goals of attackers include: exporting aspecific file out of the networked system, shutting down a specificnetwork node of the networked system, encrypting five Excel files in aspecific node of the networked system (for example as part of a demandfor ransom), and the like.

If the penetration testing campaign succeeds in achieving the goal, anattacker might be able to do the same, and therefore the networkedsystem is vulnerable to attack.

Typically, the method for an attacker to achieve the goal comprises anordered series of steps the attacker would take in order to achieve thegoal. An exemplary method for achieving a goal of controlling networknode A (e.g. the CEO's computer), illustrated in FIG. 3A, may include:

-   1. Using method Z (e.g. using a known weakness in Microsoft    Windows's networking mechanism) (step 300), attacking and    compromising network node B (e.g. the management floor    administrator's computer) (sub-goal 302).-   2. Once network node B is under the attacker's control, using method    Y (e.g. using a known method of extracting password hash codes    files) (step 304) for extracting from network node B a file    containing password hash codes of the management floor network nodes    (sub-goal 306).-   3. Based on the obtained password hash codes of the management floor    network nodes and using method X (e.g. using a known method for    recovering passwords from their hash codes) (step 308), recovering    passwords of the management floor network nodes corresponding to the    obtained password hash codes (sub-goal 310). It is assumed the CEO's    password is kept separately and is not obtained in this step.-   4. Based on the recovered passwords of the management floor network    nodes (which include the password of the CEO's personal assistant's    computer), and using method W (e.g. logging into a computer using    its known password) (step 312), attacking and compromising network    node C (e.g. the CEO's personal assistant's computer) (sub-goal    314).-   5. Once network node C (e.g. the CEO's personal assistant's    computer) is under the attacker's control, using method V (e.g.    using a known weakness in Microsoft Windows's shared folder    mechanism) (step 316) for attacking and compromising network node A    (e.g. the CEO's computer) (goal 318).

As can be seen in FIG. 3A, a way for accomplishing the goal of theattacker, also termed as “a path of attack”, includes a sequence ofattacker steps, each attacker step moving the attacker from one sub-goalto a subsequent sub-goal.

For example, attacker step 304 moves the attacker from sub-goal 302 of“controlling network node B” to sub-goal 306 of “having the passwordhash codes of the management floor network nodes”, and attacker step 308moves the attacker from sub-goal 306 of “having the password hash codesof the management floor network nodes” to sub-goal 310 of “having thepasswords of the management floor network nodes”. The first attackerstep 302 moves the attacker from the starting state, which is a dummysub-goal that is assumed to be satisfied when starting the penetrationtest, and the last attacker step 316 moves the attacker to sub-goal 318that is identical to the true goal of the attacker of the penetrationtesting campaign.

Note that FIG. 3A represents a path of attack of a networked system as adirected graph in which sub-goals and attacker steps are represented bynodes of the graph. FIG. 3B shows an alternative representation of theexemplary method shown in FIG. 3A, in which a path of attack of anetworked system is represented as a graph in which sub-goals arerepresented by nodes of the graph, while attacker steps are representedby directed edges connecting sub-goals. In the graph of FIG. 3B, thesteps and sub-goals are given the same reference numerals as in FIG. 3A.Other forms of representing paths of attack of a networked system arealso possible.

For example, paths of attack of a networked system may be represented bya list, where each member of the list corresponds to one path of attackof the networked system. Each member of the list, or each such path ofattack, is itself represented as a list, including items correspondingto all attacker steps and to all sub-goals included in that path ofattack. FIG. 3C illustrates an example of such representation, showing alist 320 including two paths of attack 322 and 324.

The representation methods of FIGS. 3A, 3B, and 3C are equivalent, andany one of them, or any other equivalent representation method, may beused for implementing the methods and systems of the present disclosure.The use of any specific representation method in any figure or anyexplanation of the present disclosure should not be construed aslimiting the scope of the claimed invention to that specificrepresentation method.

Typically, a description of the method for the attacker to achieve thegoal, which method was discovered by a penetration test, is displayed tothe user of the penetration testing system, and optionally also mailedto other people such as the administrator or CISO of the organizationowning the tested networked system.

Most penetration testing systems also provide to the user arecommendation as to how to block the discovered method of attack. Inthe above example, the recommendation may be to upgrade the MicrosoftWindows OS installed on network node B to a later version that does nothave the weakness used by step 300. Alternatively, the recommendationmay be to have the CEO's personal assistant computer, network node C,use a two-factor authentication method, so that knowledge of thepassword for network node C is insufficient for logging into thiscomputer, thus blocking step 312. Alternatively, the recommendation maybe to disable the folder sharing mechanism that is used in step 316 inthe CEO's computer, network node A, and ask the CEO and his/her personalassistant to share files using other mechanisms. In the above example,each step in the discovered method to attack results in a correspondingrecommendation for a remediation action that is aimed at blocking thatstep.

Since methods for attack might be complex and may include dozens ofattacker steps, sometimes having multiple alternatives for some of thoseattacker steps, choosing the best recommendation to provide to the userfor blocking the attack might become quite difficult. The prior artdiscloses many approaches, some of which are:

-   1. Using cost of exploitation as the determining consideration. The    cost of exploitation of an attacker step is a measure of how    difficult it is for an attacker to carry out that attacker step. For    example, the attacker step known as “ARP Spoofing” is costlier for    the attacker than an attacker step that uses a publicly available    exploit kit.    -   The cost of exploitation may be represented by a numeric score        within a given range, typically (but not necessarily) with a        higher score representing a costlier method. For example, the        given range may be [0 . . . 10], with a cost of exploitation of        ARP Spoofing being 7, and a cost of exploitation of any attacker        step using a publicly available exploit kit being 2.    -   A penetration testing system using this approach may recommend        to the user to invest in blocking the attacker step having the        lowest cost of exploitation.    -   It is noted that in the example presented above, which includes        a single and linear path of attack, it does not matter which        attacker step of the path of attack is blocked, as blocking any        attacker step protects the tested networked system from the        discovered method of attack. However, when dealing with more        complex cases, the advantage of selecting the attacker step to        block based on cost of exploitation becomes clear, as will be        evident hereinbelow.-   2. Using cost of remediation as the determining consideration. The    cost of remediation of an attacker step is a measure of how    expensive it is for the organization owning the tested networked    system to block that attacker step. For example, an attacker step    that can be blocked by simply installing a security patch for a    software application (e.g. Microsoft Word) is much less costly to    fix than an attacker step that requires buying and installing a new    router in order to split an existing sub-network into two different    sub-networks.    -   The cost of remediation may be represented by a numeric score        within a given range, typically (but not necessarily) with a        higher score representing a costlier method. For example, the        given range may be [0 . . . 10], with a cost of remediation        requiring only installing a patch being 1, and a cost of        remediation requiring a new router being 8.    -   A penetration testing system using this approach may recommend        to the user to invest in blocking the attacker step having the        lowest cost of remediation, thus blocking an attacker from        achieving its goal while investing the smallest budget.-   3. Using probability of success as the determining consideration.    The probability of success of an attacker step is a measure of how    probable is it that execution of the attacker step by the attacker    will succeed in achieving the sub-goal that the attacker step is    intended to achieve, taking into account the currently available    knowledge regarding the state of the attacked networked system. For    example, an attacker step that is based on exploiting a known    Windows 7 vulnerability may have high probability of success when    applied to a network node having the original version of the OS    installed, and may have a low probability of success when applied to    a network node in which a certain security patch has been installed.    -   Typically, probabilities of success are expressed in percentages        in the range of 0% to 100%. Alternatively, the probabilities of        success may be represented by numeric values in the range of        zero to one, where zero corresponds to 0% and one corresponds to        100%. However, any other numerical scale may be used for        representing probabilities of success, provided that the scale        is a monotonically increasing or monotonically decreasing        function of how probable is it that the attacker step will        succeed in achieving its sub-goal.    -   A penetration testing system using this approach may recommend        to the user to invest in blocking the attacker step having the        highest probability of success.-   4. Using a combination of any of the previous considerations. For    example, a combination of cost of exploitation and cost of    remediation may be used as the determining considerations.    -   A function of both factors is defined, and the recommendation of        which attacker step to block is selected based on values of the        function for the various attacker steps of the path of attack.    -   For example, the value of the function for each attacker step        may be defined to be the multiplication of the cost of        exploitation of the attacker step by the cost of remediation of        the attacker step, and the recommendation may be to block the        attacker step having the lowest value of the function.    -   In another example, the value of the function for each attacker        step may be defined as follows—if the cost of exploitation of        the attacker step is lower than five, then the value of the        function is the cost of remediation multiplied by three, and if        the cost of exploitation of the attacker step is equal to or        greater than five, then the value of the function is the cost of        remediation multiplied by seven. Again, the recommendation may        be to block the attacker step having the lowest value of the        function.

The example of a method for achieving an attacker's goal of compromisinga networked system illustrated in FIGS. 3A and 3B has a relativelysimple attacker's goal and a single and relatively simple path of attackfor the attacker to achieve his goal. Consequently, deciding whichrecommendation to provide to the user is also a relatively simple task.

In other cases, the attacker's goal and the options available for theattacker to achieve that goal may be more complex, and consequently theselection of a suitable recommendation to provide to the user is alsomore complex. For example, the attacker's goal may be to control eitherone of the CEO's computer and the CFO's computer. In this case, themethod to achieve the goal contains two separate “branches” (i.e. pathsof attack), merging in the final sub-goal (see FIG. 4, which isequivalent to FIG. 3C, but is presented as a graph rather than as alist).

One branch of the graph of FIG. 4 is identical to the graph illustratedin FIG. 3A, and steps and sub-goals thereof are indicated by the samereference numerals, while the second branch includes the following stepsand sub-goals:

-   1. Using method U (e.g. using a known weakness in Microsoft    Windows's RPC mechanism) (step 400), attacking and compromising    network node D (e.g. the finance floor administrator's computer)    (sub-goal 402).-   2. Once network node D is under the attacker's control, using method    T (e.g. using a known method of extracting password files) (step    404) for extracting from network node D a file containing passwords    of the finance floor network nodes (sub-goal 406). It is assumed the    CFO's password is kept separately and is not obtained in this step.-   3. Based on the obtained passwords of the finance floor network    nodes (which include the password of the CFO's personal assistant's    computer), and using method S (e.g. logging into a computer using    its known password) (step 408), attacking and compromising network    node E (e.g. the CFO's personal assistant's computer) (sub-goal    410).-   4. Once network node E (the CFO's personal assistant's computer) is    under the attacker's control, using method R (e.g. using a known    weakness in Microsoft Windows's file sharing mechanism) (step 412)    for attacking and compromising network node F (e.g. the CFO's    computer) (sub-goal 414).

It is clear that in this case it is not possible to block allpossibilities of attack by blocking a single attacker step. Each of thetwo branches has to be separately blocked, requiring blocking of atleast one attacker step in each branch.

A possible algorithm for selecting recommendations of steps to beblocked in this case may operate as follows:

-   A. Determine the best attacker step to block in a first branch of    the two, assuming the other branch does not exist.-   B. Determine the best attacker step to block in the other branch,    assuming the first branch does not exist.-   C. Recommend blocking both of the steps determined in A and B.

Blocking one attacker step in each of the two branches guarantees thatboth branches become unusable (or at least less usable) for theattacker. The step recommended for blocking the first branch and thestep recommended for blocking the second branch may be determined usingany method known in the art—based on cost of exploitation, based on costof remediation, based on both cost of exploitation and cost ofremediation, etc. It should be noted that the determination for each ofthe branches is completely independent of the determination for theother branch. Moreover, the method of determination may be different forthe two branches—for example, in one branch the determination may bebased on cost of remediation, while in the other branch thedetermination may be based on a combination of cost of remediation andprobability of success.

Blocking one attacker step in each path of attack is useful when theowner of the tested networked system blocks as many attacker steps asare required for completely blocking all possibilities for the attackersto compromise the networked system. However, this is not always the casein the real world. Remediation actions recommended for blocking allpossibilities of attack might be highly expensive and their combinedcosts might exceed the budget available for protecting the networkedsystem. For example, blocking a certain attacker step may requireinstalling a certain defensive application on each node of the networkedsystem. Even for an inexpensive application costing just $10 per node,in a large network of 10,000 network nodes this translates to a directcost of $100,000, prior to training of users and other indirect costs.

Therefore, some penetration testing systems provide theirrecommendations as a list, ordered by priority. This way the availablefunds can be allocated to implementing the recommendations at the top ofthe list, which are the most urgent and/or important to fix. Even thoughsuch partial implementation of the recommendations list might notprovide a fool-proof solution, leaving some unblocked attack pathsavailable to potential attackers, it nevertheless makes the best use ofthe available funds and improves the security posture of the networkedsystem to the maximal possible under the existing budget constraints.

Algorithms for selecting the highest priority recommendation may operatein the following high-level way:

-   -   1. Determine the list of “branches” (i.e. paths of attack)        available for a potential attacker to achieve his goal.    -   2. If there is only one branch, define it to be the “critical        branch”.    -   3. Otherwise, if there are multiple branches, evaluate each of        the branches according to some criteria and generate a numerical        score for each branch. Based on generated numerical scores for        all the branches, define one of the branches to be the “critical        branch”.    -   4. Determine, based on some criteria, one of the attacker steps        of the critical branch to be the best recommendation.

The following is an example of such an algorithm:

-   -   1. Determine the list of “branches” (i.e. paths of attack)        available for a potential attacker to achieve his goal.    -   2. If there is only one branch, define it to be the “critical        branch”.    -   3. Otherwise, if there are multiple branches:        -   a. For each branch, calculate a “cost of exploitation”            score. The cost of exploitation score of a branch is the sum            of the costs of exploitation of all the attacker steps            included in the branch.        -   b. The branch having the lowest cost of exploitation score            of all branches is defined to be the “critical branch”.    -   4. Determine which attacker step of the critical branch has the        lowest cost of remediation.

This step is recommended as the highest priority attacker step to beblocked.

FIG. 5 demonstrates how the exemplary algorithm described above may beapplied to the method of attack illustrated in FIG. 4. In FIG. 5, thecost of exploitation and cost of remediation of each attacker step areprovided. The format used in FIG. 5 for showing the costs is “X/Y”,where “X” is the cost of exploitation according to some exploitationcost scale and “Y” is the cost of remediation according to someremediation cost scale.

As there are two branches in FIG. 5 identified in step 1 of thealgorithm, the implementation of the algorithm skips to step 3, wherethe critical branch must be identified. The cost of exploitation for theleft-side branch is 2+4+7+1+6=20. The cost of exploitation for theright-side branch is 6+7+1+8=22. The branch having the lowest cost ofexploitation is the left-side branch, and in accordance with step 3b ofthe algorithm the left-side branch is determined to be the criticalbranch. Within the left-side branch, the attacker step having the lowestcost of remediation is step 300 of applying method Z. Therefore, thealgorithm decides that the highest priority attacker step to be blockedis step 300, as the method determined it to be the most cost-effectiveinvestment. It should be noted that even after blocking that attackerstep, the networked system is still not fully protected—an attackermight still use the route of the right-side branch to compromise it, buthis cost of exploitation would be somewhat higher after the remediationof blocking attacker step 300.

FIG. 6 provides another example of use of the exemplary algorithm. Thepaths of attack of FIG. 6 are nearly identical to those of FIG. 5, withthe only difference being that in FIG. 6 step 408 (the third step of theright-side branch) “Use method S” is replaced with another copy of step312, “Use method W”, where methods S and W of steps 408 and 312 have thesame costs of exploitation and remediation. Note that in FIG. 6, step312 of “Use method W” appears in both branches. The example of FIG. 6may correspond to a case in which the CEO and the CFO share a commonpersonal assistant. Applying the above algorithm to FIG. 6 results inthe same recommendation provided in FIG. 5—the highest priority attackerstep to be blocked is again step 300, the first step in the left-sidebranch, leaving an attacker the option to attack using the right-sidebranch.

However, a careful review of FIG. 6 indicates that blocking of attackerstep 300 is clearly not the most cost-effective move. By blockingattacker step 312, “Use method W”, both branches can be blocked by asingle remediation action. The cost of that single remediation actionmay be higher than the cost of blocking step 300 (in this example, 4 vs.3), but the benefit gained from this slight additional cost issignificant—instead of just somewhat increasing the cost of exploitationfor the attacker while leaving paths of attack available to theattacker, blocking step 312 would completely block the attacker'sability to obtain his goal in compromising the networked system.

The flaw of the previous method, which causes it to recommend anon-optimal solution, is that it focuses on the “branches of attack”(i.e. on complete paths of attack) as fundamental and independentbuilding blocks, ignoring connections or relations existing betweendifferent branches at a lower level.

There was thus a need in the art to have a method for prioritizingremediation recommendations so as to provide cost-effectiverecommendations even when there are dependencies between different pathsof attack.

U.S. Pat. No. 10,382,473 addresses the above need and discloses methodsthat focus on single attacker steps (instead of complete branches ofattack), examining how blocking each single attacker step affects thevulnerability of the tested networked system. For each attacker stepincluded in at least one path of attack determined to exist in thetested networked system, the methods of the '063 application compute acorresponding vulnerability grade. The vulnerability grade of a givenattacker step is determined according to a vulnerability score computedfor the tested networked system when assuming that the given attackerstep is being blocked. The remediation recommendations are then selectedto be the blocking of the attacker step(s) associated with the highestvulnerability to the tested networked system.

While the methods disclosed in the '063 application provide more optimalrecommendation than prior methods, they are relatively complex andcomputation-intensive.

There is thus a need in the art to have a method for prioritizingremediation recommendations so as to provide cost-effectiverecommendations even when there are dependencies between different pathsof attack, while using relatively simple methods to identify theremediation recommendations.

SUMMARY OF THE INVENTION

Some embodiments of the invention relate to methods and systems forcarrying out automated penetration testing, and to providing to the userrecommendations for the highest priority sub-goals to protect in orderto improve the security of the tested networked system againstattackers.

According to an embodiment of a first aspect of the invention, there isprovided a method for providing, by a penetration testing system, arecommendation for improving the security of a networked system againstattackers, the method including:

-   a. carrying out one or more tests of the networked system by the    penetration testing system;-   b. based on results of the one or more tests of the networked    system, determining multiple paths of attack available to the    attackers, each path of attack of the determined multiple paths of    attack being an ordered sequence of one or more attacker steps and    one or more sub-goals;-   c. assigning a calculated importance score to each of multiple    sub-goals, wherein (i) each sub-goal of the multiple sub-goals is    included in at least one of the determined multiple paths of attack,    and (ii) for at least one given sub-goal of the multiple sub-goals,    the importance score assigned to the given sub-goal is based on a    number of paths of attack of the determined multiple paths of attack    which include the given sub-goal;-   d. selecting one sub-goal included in at least one of the determined    multiple paths of attack, the selecting of the one sub-goal being    based on the importance score assigned to at least one of the    multiple sub-goals; and-   e. providing a recommendation to protect the selected one sub-goal,    the providing of the recommendation including at least one operation    selected from the group consisting of:    -   i. causing a display device to display information about the        recommendation;    -   ii. recording the information about the recommendation in a        file; and    -   iii. electronically transmitting the information about the        recommendation.

In some embodiments, for each given sub-goal of the multiple sub-goals,the importance score assigned to the given sub-goal is based on thenumber of paths of attack of the determined multiple paths of attackwhich include the given sub-goal.

In some embodiments, for at least one given sub-goal of the multiplesub-goals, the importance score assigned to the given sub-goal is basedonly on the number of paths of attack of the determined multiple pathsof attack which include the given sub-goal.

In some embodiments, for each given sub-goal of the multiple sub-goals,the importance score assigned to the given sub-goal is based only on thenumber of paths of attack of the determined multiple paths of attackwhich include the given sub-goal.

In some embodiments, for at least one given sub-goal of the multiplesub-goals, the importance score assigned to the given sub-goal is equalto the number of paths of attack of the determined multiple paths ofattack which include the given sub-goal.

In some embodiments, the selecting of the one sub-goal included in atleast one of the determined multiple paths of attack includes selectingone sub-goal whose assigned importance score meets a predefinedcriterion.

In some embodiments, the method further includes representing thedetermined multiple paths of attack by a graph, where each given path ofattack of the determined multiple paths of attack corresponds to a pathin the graph.

In some such embodiments, the representing includes, for each given pathin the graph corresponding to a specific path of attack of thedetermined multiple paths of attack, representing all sub-goals and allattacker steps included in the specific path of attack as graph nodesincluded in the given path in the graph.

In some other such embodiments, the representing includes, for eachgiven path in the graph corresponding to a specific path of attack ofthe determined multiple paths of attack, representing all sub-goalsincluded in the specific path of attack as graph nodes included in thegiven path in the graph and representing all attacker steps included inthe specific path of attack as graph edges included in the given path inthe graph.

In some embodiments, the method further includes representing thedetermined multiple paths of attack by a list, where each given path ofattack of the determined multiple paths of attack corresponds to an itemin the list that includes all sub-goals and all attacker steps includedin the given path of attack.

In some embodiments, each given path of attack of the determinedmultiple paths of attack starts at a starting sub-goal that is assumedto be achievable by the attackers and ends at a final sub-goal which isassumed to be a goal of the attackers in at least one test of the one ormore tests.

In some embodiments, for each given path of attack of the determinedmultiple paths of attack: (A) each attacker step included in the givenpath of attack is preceded by a sub-goal and followed by a sub-goal inthe given path of attack, (B) each sub-goal included in the given pathof attack, except for a starting sub-goal of the given path of attack,is preceded by an attacker step in the given path of attack, and (C)each sub-goal included in the given path of attack, except for a finalsub-goal of the given path of attack, is followed by an attacker step inthe given path of attack.

In some embodiments, for at least one given sub-goal of the multiplesub-goals, the importance score assigned to the given sub-goal is basedon (i) the number of paths of attack of the determined multiple paths ofattack which include the given sub-goal, and (ii) at least one memberselected from the group consisting of: a cost of remediation of thegiven sub-goal, a cost of exploitation of an attacker step leading tothe given sub-goal, a cost of remediation of an attacker step leading tothe given sub-goal, and a probability of success of an attacker stepleading to the given sub-goal.

In some embodiments, the importance scores assigned to the multiplesub-goals are such that a higher numerical value of an importance scoreof a specific sub-goal is indicative of a higher degree of importanceassociated with the specific sub-goal.

In some other embodiments, the importance scores assigned to themultiple sub-goals are such that a lower numerical value of animportance score of a specific sub-goal is indicative of a higher degreeof importance associated with the specific sub-goal.

In some embodiments, the selecting of the one sub-goal includesselecting, as the one sub-goal, a sub-goal whose assigned importancescore has the highest numerical value of all importance scores assignedto sub-goals included in at least one of the determined multiple pathsof attack.

In some embodiments, the selecting of the one sub-goal includesselecting, as the one sub-goal, a sub-goal whose assigned importancescore has the lowest numerical value of all importance scores assignedto sub-goals included in at least one of the determined multiple pathsof attack.

In some embodiments, the selecting of the one sub-goal includes: inresponse to at least two sub-goals of the multiple sub-goals havingassigned a common importance score, which common importance score is animportance score indicative of the highest degree of importance of allthe importance scores assigned to the multiple sub-goals, using atie-breaking rule to select one of the at least two sub-goals havingassigned the common importance score as the one sub-goal.

In some such embodiments, the tie-breaking rule depends on at least onemember selected from the group consisting of: a cost of remediation ofone of the at least two sub-goals having assigned the common importancescore, a cost of exploitation of an attacker step leading to one of theat least two sub-goals having assigned the common importance score, acost of remediation of an attacker step leading to one of the at leasttwo sub-goals having assigned the common importance score, and aprobability of success of an attacker step leading to one of the atleast two sub-goals having assigned the common importance score.

In some other such embodiments, the using of the tie-breaking ruleincludes randomly selecting one of the at least two sub-goals havingassigned the common importance score.

According to an embodiment of the first aspect of the invention, thereis provided a system for providing a recommendation for improving thesecurity of a networked system against attackers, the system including:

-   a. a penetration-testing-campaign module including:    -   i. one or more penetration-testing-campaign processors; and    -   ii. a penetration-testing-campaign non-transitory computer        readable storage medium for instructions execution by the one or        more penetration-testing-campaign processors, the        penetration-testing-campaign non-transitory computer readable        storage medium having stored instructions to carry out one or        more penetration tests of the networked system;-   b. a sub-goal-selection module including:    -   i. one or more sub-goal-selection processors; and    -   ii. a sub-goal-selection non-transitory computer readable        storage medium for instructions execution by the one or more        sub-goal-selection processors, the sub-goal-selection        non-transitory computer readable storage medium having stored:        -   1) instructions to receive, from the            penetration-testing-campaign module, results of the one or            more penetration tests of the networked system;        -   2) instructions to determine, based on the received results,            multiple paths of attack available to the attackers, each            path of attack of the determined multiple paths of attack            being an ordered sequence of one or more attacker steps and            one or more sub-goals;        -   3) instructions to assign a calculated importance score to            each of multiple sub-goals, wherein (i) each sub-goal of the            multiple sub-goals is included in at least one of the            determined multiple paths of attack, and (ii) for at least            one given sub-goal of the multiple sub-goals, the importance            score assigned to the given sub-goal is based on a number of            paths of attack of the determined multiple paths of attack            which include the given sub-goal; and        -   4) instructions to select one sub-goal included in at least            one of the determined multiple paths of attack, the            selecting of the one sub-goal being based on the importance            score assigned to at least one of the multiple sub-goals;            and-   c. a reporting module including:    -   i. one or more reporting processors; and    -   ii. a reporting non-transitory computer readable storage medium        for instructions execution by the one or more reporting        processors, the reporting non-transitory computer readable        storage medium having stored:        -   1) instructions to receive, from the sub-goal-selection            module, an identification of the selected one sub-goal; and        -   2) instructions to provide a recommendation to protect the            selected one sub-goal, the instructions to provide the            recommendation including at least one member selected from            the group consisting of:            -   I. instructions to cause a display device to display                information about the recommendation;            -   II. instructions to record the information about the                recommendation in a file; and            -   III. instructions to electronically transmit the                information about the recommendation.

According to an embodiment of a second aspect of the invention, there isprovided a method for providing, by a penetration testing system, arecommendation for improving the security of a networked system againstattackers, the method including:

-   a. initializing a list of sub-goals that should be protected to be    an empty list;-   b. obtaining a halting condition, the halting condition including a    Boolean condition applied to the list of sub-goals;-   c. carrying out one or more tests of the networked system by the    penetration testing system;-   d. based on results of the one or more tests of the networked    system, determining multiple paths of attack available to the    attackers, each path of attack of the determined multiple paths of    attack being an ordered sequence of one or more attacker steps and    one or more sub-goals;-   e. initializing a group of relevant paths of attack to consist of    the determined multiple paths of attack;-   f. assigning a calculated importance score to each of one or more    sub-goals, wherein (i) each sub-goal of the one or more sub-goals is    included in at least one path of attack included in the group of    relevant paths of attack, and (ii) for at least one given sub-goal    of the one or more sub-goals, the importance score assigned to the    given sub-goal is based on a number of paths of attack in the group    of relevant paths of attack which include the given sub-goal;-   g. selecting one sub-goal included in at least one member of the    group of relevant paths of attack and adding the one sub-goal to the    list of sub-goals, the selecting of the one sub-goal being based on    the importance scores assigned to at least one of the one or more    sub-goals;-   h. modifying the group of relevant paths of attack by removing from    it every path of attack that includes the one sub-goal;-   i. evaluating the halting condition for the list of sub-goals;-   j. in response to determining that (i) the halting condition is not    satisfied, and (ii) the group of relevant paths of attack is not    empty, repeating steps f to j; and-   k. in response to determining that (i) the halting condition is    satisfied, or (ii) the group of relevant paths of attack is empty,    providing a recommendation to protect one or more sub-goals from the    list of sub-goals, the providing of the recommendation including at    least one operation selected from the group consisting of:    -   i. causing a display device to display information about the        recommendation;    -   ii. recording the information about the recommendation in a        file; and    -   iii. electronically transmitting the information about the        recommendation.

In some embodiments, for each given sub-goal of the one or moresub-goals, the importance score assigned to the given sub-goal is basedon the number of paths of attack in the group of relevant paths ofattack which include the given sub-goal.

In some embodiments, for at least one given sub-goal of the one or moresub-goals, the importance score assigned to the given sub-goal is basedonly on the number of paths of attack in the group of relevant paths ofattack which include the given sub-goal.

In some embodiments, for each given sub-goal of the one or moresub-goals, the importance score assigned to the given sub-goal is basedonly on the number of paths of attack in the group of relevant paths ofattack which include the given sub-goal.

In some embodiments, for at least one given sub-goal of the one or moresub-goals, the importance score assigned to the given sub-goal is equalto the number of paths of attack in the group of relevant paths ofattack which include the given sub-goal.

In some embodiments, the selecting of the one sub-goal included in atleast one member of the group of relevant paths of attack includesselecting one sub-goal whose assigned importance score meets apredefined criterion.

In some embodiments, the method further includes representing thedetermined multiple paths of attack by a graph, where each given path ofattack of the determined multiple paths of attack corresponds to a pathin the graph.

In some such embodiments, the representing includes, for each given pathin the graph corresponding to a specific path of attack of thedetermined multiple paths of attack, representing all sub-goals and allattacker steps included in the specific path of attack as graph nodesincluded in the given path in the graph.

In some other such embodiments, the representing includes, for eachgiven path in the graph corresponding to a specific path of attack ofthe determined multiple paths of attack, representing all sub-goalsincluded in the specific path of attack as graph nodes included in thegiven path in the graph and representing all attacker steps included inthe specific path of attack as graph edges included in the given path inthe graph.

In some embodiments, the method further includes representing thedetermined multiple paths of attack by a list, where each given path ofattack of the determined multiple paths of attack corresponds to an itemin the list that includes all sub-goals and all attacker steps includedin the given path of attack.

In some embodiments, each given path of attack of the determinedmultiple paths of attack starts with a starting sub-goal that is assumedto be achievable by the attackers and ends with a final sub-goal whichis assumed to be a goal of the attackers in at least one test of the oneor more tests.

In some embodiments, for each given path of attack of the determinedmultiple paths of attack: (A) each attacker step included in the givenpath of attack is preceded by a sub-goal and followed by a sub-goal inthe given path of attack, (B) each sub-goal included in the given pathof attack, except for a starting sub-goal of the given path of attack,is preceded by an attacker step in the given path of attack, and (C)each sub-goal included in the given path of attack, except for a finalsub-goal of the given path of attack, is followed by an attacker step inthe given path of attack.

In some embodiments, for at least one given sub-goal of the one or moresub-goals, the importance score assigned to the given sub-goal is basedon (i) the number of paths of attack in the group of relevant paths ofattack which include the given sub-goal, and (ii) at least one memberselected from the group consisting of: a cost of remediation of thegiven sub-goal, a cost of exploitation of an attacker step leading tothe given sub-goal, a cost of remediation of an attacker step leading tothe given sub-goal, and a probability of success of an attacker stepleading to the given sub-goal.

In some embodiments, the importance scores assigned to the one or moresub-goals are such that a higher numerical value of an importance scoreof a specific sub-goal is indicative of a higher degree of importanceassociated with the specific sub-goal.

In some other embodiments, the importance scores assigned to the one ormore sub-goals are such that a lower numerical value of an importancescore of a specific sub-goal is indicative of a higher degree ofimportance associated with the specific sub-goal.

In some embodiments, the selecting of the one sub-goal includesselecting, as the one sub-goal, a sub-goal whose assigned importancescore has the highest numerical value of all sub-goals included in atleast one path of attack in the group of relevant paths of attack.

In some embodiments, the selecting of the one sub-goal includesselecting, as the one sub-goal, a sub-goal whose assigned importancescore has the lowest numerical value of all sub-goals included in atleast one path of attack in the group of relevant paths of attack.

In some embodiments, the selecting of the one sub-goal includes: inresponse to at least two sub-goals of the one or more sub-goals havingassigned a common importance score, which common importance score is animportance score indicative of the highest degree of importance of allthe importance scores assigned to all sub-goals included in at least onepath of attack in the group of relevant paths of attack, using atie-breaking rule to select one of the at least two sub-goals havingassigned the common importance score as the one sub-goal.

In some such embodiments, the tie-breaking rule depends on at least onemember selected from the group consisting of: a cost of remediation ofone of the multiple sub-goals having assigned the common importancescore, a cost of exploitation of an attacker step leading to one of theat least two sub-goals having assigned the common importance score, acost of remediation of an attacker step leading to one of the at leasttwo sub-goals having assigned the common importance score, and aprobability of success of an attacker step leading to one of the atleast two sub-goals having assigned the common importance score.

In some other such embodiments, the using of the tie-breaking ruleincludes randomly selecting one of the multiple sub-goals havingassigned the common importance score.

In some embodiments, (i) the list of sub-goals is an ordered list ofsub-goals, (ii) the adding of the one sub-goal to the list of sub-goalsincludes adding the one sub-goal at an end of the ordered list ofsub-goals such that the one sub-goal becomes a last member of theordered list of sub-goals, and (iii) the providing a recommendation toprotect one or more sub-goals from the list of sub-goals includesproviding a recommendation to protect the one or more sub-goals from thelist of sub-goals according to an order of the one or more sub-goals inthe ordered list of sub-goals.

In some embodiments, the halting condition is true if and only if thelist of sub-goals consists of one sub-goal.

In some embodiments, the halting condition is true if and only if thelist of sub-goals consists of a pre-determined number of sub-goals.

In some embodiments, the halting condition is true if and only if a sumof remediation costs of all members of the list of sub-goals satisfies asecond Boolean condition.

In some embodiments, the second Boolean condition is true if and only ifthe sum of remediation costs of all members of the list of sub-goals ishigher than a pre-determined threshold.

In some embodiments, the second Boolean condition is true if and only ifthe sum of remediation costs of all members of the list of sub-goals ishigher than or equal to a pre-determined threshold.

In some embodiments, the second Boolean condition is true if and only ifthe sum of remediation costs of all members of the list of sub-goals islower than a pre-determined threshold.

In some embodiments, the second Boolean condition is true if and only ifthe sum of remediation costs of all members of the list of sub-goals islower than or equal to a pre-determined threshold.

According to an embodiment of the second aspect of the invention, thereis provided a system for providing a recommendation for improving thesecurity of a networked system against attackers, the system including:

-   a. a penetration-testing-campaign module including:    -   i. one or more penetration-testing-campaign processors; and    -   ii. a penetration-testing-campaign non-transitory computer        readable storage medium for instructions execution by the one or        more penetration-testing-campaign processors, the        penetration-testing-campaign non-transitory computer readable        storage medium having stored instructions to carry out one or        more penetration tests of the networked system;-   b. a sub-goals-selection module including:    -   i. one or more sub-goals-selection processors; and    -   ii. a sub-goals-selection non-transitory computer readable        storage medium for instructions execution by the one or more        sub-goals-selection processors, the sub-goals-selection        non-transitory computer readable storage medium having stored:        -   1) first instructions to initialize a list of sub-goals that            should be protected to be an empty list;        -   2) second instructions to obtain a halting condition, the            halting condition including a Boolean condition applied to            the list of sub-goals;        -   3) third instructions to receive, from the            penetration-testing-campaign module, results of the one or            more penetration tests of the networked system;        -   4) fourth instructions to determine, based on the received            results of the one or more tests of the networked system,            multiple paths of attack available to the attackers, each            path of attack of the determined multiple paths of attack            being an ordered sequence of one or more attacker steps and            one or more sub-goals;        -   5) fifth instructions to initialize a group of relevant            paths of attack to consist of the determined multiple paths            of attack;        -   6) sixth instructions to assign a calculated importance            score to each of one or more sub-goals, wherein (i) each            sub-goal of the one or more sub-goals is included in at            least one path of attack included in the group of relevant            paths of attack, and (ii) for at least one given sub-goal of            the one or more sub-goals, the importance score assigned to            the given sub-goal is based on a number of paths of attack            in the group of relevant paths of attack which include the            given sub-goal;        -   7) seventh instructions to select one sub-goal included in            at least one member of the group of relevant paths of attack            and to add the one sub-goal to the list of sub-goals, the            selecting of the one sub-goal being based on the importance            scores assigned to at least one of the one or more            sub-goals;        -   8) eighth instructions to modify the group of relevant paths            of attack by removing from it every path of attack that            includes the one sub-goal;        -   9) ninth instructions to evaluate the halting condition for            the list of sub-goals;        -   10) tenth instructions, to be carried out in response to            determining that (i) the halting condition is not satisfied,            and (ii) the group of relevant paths of attack is not empty,            to repeat the sixth instructions to the tenth instructions;            and        -   11) eleventh instructions, to be carried out in response to            determining that (i) the halting condition is satisfied,            or (ii) the group of relevant paths of attack is empty, to            select one or more sub-goals from the list of sub-goals; and-   c. a reporting module including:    -   i. one or more reporting processors; and    -   ii. a reporting non-transitory computer readable storage medium        for instructions execution by the one or more reporting        processors, the reporting non-transitory computer readable        storage medium having stored:        -   1) instructions to receive, from the sub-goals-selection            module, an identification of the one or more selected            sub-goals; and        -   2) instructions to provide a recommendation to protect the            one or more selected sub-goals, the instructions to provide            the recommendation including at least one member selected            from the group consisting of:            -   I. instructions to cause a display device to display                information about the recommendation;            -   II. instructions to record the information about the                recommendation in a file; and            -   III. instructions to electronically transmit the                information about the recommendation.

All technical and scientific terms used herein have the same meaning ascommonly understood by one of ordinary skill in the art to which theinvention pertains, unless explicitly defined in this application. Incase of conflict, the specification, including definitions, will takeprecedence.

As used herein, the terms “comprising”, “including”, “having” andgrammatical variants thereof are to be taken as specifying the statedfeatures, integers, steps or components but do not preclude the additionof one or more additional features, integers, steps, components orgroups thereof. These terms encompass the terms “consisting of” and“consisting essentially of”.

In the summary and in the description that follows, the terms“importance score” and “importance measure” are used interchangeably,and have the same meaning.

BRIEF DESCRIPTION OF THE FIGURES

The invention is herein described, by way of example only, withreference to the accompanying drawings. With specific reference now tothe drawings in detail, it is stressed that the particulars shown are byway of example and for purposes of illustrative discussion of thepreferred embodiments of the present invention only, and are presentedin the cause of providing what is believed to be the most useful andreadily understood description of the principles and conceptual aspectsof the invention. In this regard, no attempt is made to show structuraldetails of the invention in more detail than is necessary for afundamental understanding of the invention, the description taken withthe drawings making apparent to those skilled in the art how the severalforms of the invention may be embodied in practice. Throughout thedrawings, like-referenced characters are used to designate likeelements.

In the drawings:

FIG. 1A (PRIOR ART) is a block diagram of code modules of a typicalpenetration testing system;

FIG. 1B (PRIOR ART) is a flow-chart related to the system of FIG. 1A;

FIG. 2 (PRIOR ART) illustrates a prior art computing device;

FIGS. 3A and 3B are different graph representations of an example of amethod for an attacker to achieve a goal of an attack, the examplehaving a single branch of attack;

FIG. 3C is a list representation of an example of a method for anattacker to achieve a goal of an attack, the example having multiplebranches of attack;

FIG. 4 is a graph representation of an example of a method for anattacker to achieve a goal of an attack, the example having multiplebranches of attack;

FIG. 5 is a representation of the paths of attack of FIG. 4, havingcosts of remediation and of exploitation associated with attacker steps;

FIG. 6 is a graph representation of another example of a method for anattacker to achieve a goal of an attack, the example having multiplebranches of attack that share a common attacker step;

FIGS. 7A, 7B, and 7C are equivalent graph representations of the samepaths of attack, having costs of remediation and of exploitationassociated with attacker steps;

FIG. 8 is a graph representation of the paths of attack of FIG. 7A,following application thereto of a first iteration of a method of thepresent invention;

FIGS. 9A and 9B, together, are a graph representation of another exampleof a method of an attacker to achieve a goal, the example havingmultiple branches of attack that include multiple occurrences of similarattacker steps;

FIG. 10A is a schematic block diagram of a system for providing arecommendation for improving the security of a network system againstattackers according to an embodiment of a first aspect of the invention;

FIG. 10B is a schematic block diagram of a sub-goal selection moduleforming part of the system of FIG. 10A;

FIG. 11 is a flow chart of a method for providing a recommendation forimproving the security of a network system against attackers accordingto an embodiment of the first aspect of the invention;

FIG. 12A is a schematic block diagram of a system for providing arecommendation for improving the security of a network system againstattackers according to an embodiment of a second aspect of theinvention;

FIGS. 12B and 12C, together, are a schematic block diagram of asub-goals selection module forming part of the system of FIG. 12A; and

FIGS. 13A and 13B, together, are a flow chart of a method for providinga recommendation for improving the security of a network system againstattackers according to an embodiment of the second aspect of theinvention.

DESCRIPTION OF SOME EMBODIMENTS OF THE INVENTION

The invention, in some embodiments, relates to penetration testing of anetworked system, and specifically to methods and systems for providingto a user optimal remediation recommendations for protecting sub-goalsin order to block paths of attack identified during penetration testing.

The present disclosure should be interpreted according to thedefinitions in the “Definitions Section” at the end of thespecification. In case of a contradiction between the definitions in the“Definitions Section” at the end of the specification and other sectionsof this disclosure, the “Definitions Section” at the end of thespecification section should prevail.

In case of a contradiction between the “Definitions Section” at the endof the specification and a definition or a description in any otherdocument, including in another document incorporated in this disclosureby reference, the “Definitions Section” at the end of the specificationshould prevail, even if the definition or the description in the otherdocument is commonly accepted by a person of ordinary skill in the art.

The present invention provides a solution to the challenges discussedhereinabove with respect to the prior art, and specifically providesmethods and systems for efficiently prioritizing sub-goals to beprotected such that cost-effective recommendations are provided to theuser even when there are dependencies between different paths of attack.In the general case, the output of the method is a priority-ordered listof sub-goals that are recommended to be protected by suitableremediation actions.

The proposed solution operates at the level of individual sub-goalsrather than complete paths of attack or individual attacker steps.Consequently, implementing the proposed solution requires defining of ameasure of importance for each of the sub-goals included in at least onepath of attack known to be available to attackers attacking thenetworked system.

The measure of importance used for grading the importance of sub-goalscan be such that the higher the numerical value of the level ofimportance, the more important is the sub-goal. Alternatively, themeasure of importance used for grading the importance of sub-goals canbe such that the higher the numerical value of the level of importance,the less important is the sub-goal.

For the purpose of the methods of the proposed solution, the startingsub-goal and the final sub-goal of a path of attack are ignored and arenot evaluated for importance. Consequently, in this disclosure the term“true sub-goal of a path of attack” means a sub-goal included in thepath of attack that is not the starting sub-goal of the path of attackor the final sub-goal of the path of attack.

A first example for a measure of importance of a given sub-goal is thenumber of paths of attack in which the given sub-goal is included. Thehigher the number, the more important is the sub-goal. The reasoningbehind using this measure is that protecting a sub-goal included in manypaths of attack results in blocking many paths of attack.

In the example of FIG. 7A, there are three paths of attack, eachincluding three true sub-goals (that is—three sub-goals that are not theinitial or final sub-goals). However, some of the sub-goals appear inmore than one of the three paths of attack. As such, the number ofdistinct true sub-goals in FIG. 7A is six, not nine.

As can be seen in FIG. 7A:

-   -   A. Sub-goal 1 appears in one path of attack, and consequently        its importance measure is 1;    -   B. Sub-goal 2 appears in one path of attack, and consequently        its importance measure is 1;    -   C. Sub-goal 3 appears in three paths of attack, and consequently        its importance measure is 3;    -   D. Sub-goal 4 appears in one path of attack, and consequently        its importance measure is 1;    -   E. Sub-goal 5 appears in two paths of attack, and consequently        its importance measure is 2; and    -   F. Sub-goal 6 appears in one path of attack, and consequently        its importance measure is 1.

Therefore, the most important sub-goal in FIG. 7A is sub-goal 3(importance measure 3), followed by sub-goal 5 (importance measure 2),followed by all other sub-goals, all of which have an equal and lowestimportance (importance measure 1).

A second example for a measure of importance of a given sub-goal is themultiplication of the number of paths of attack in which the givensub-goal is included by the lowest cost of exploitation of all attackersteps leading to an occurrence of the given sub-goal in a path ofattack. As in the previous example of an importance measure, the highestthe number, the more important is the sub-goal.

The reasoning behind using this measure is similar to the reasoningbehind using the importance measure of the first example. However, inthis second example, there is an additional aspect of “weightingfactors” that take into account the ease or difficulty with which anattacker can reach a given sub-goal. In order for this exemplaryimportance measure to make sense, the definition of the cost ofexploitation in this example should be such that the easier it is for anattacker to carry out an attacker step, the higher is the numericalvalue of the cost of exploitation. This implies that the “lowest cost ofexploitation” used in the calculation is the numerically highest cost ofexploitation of all attacker steps leading to an occurrence of the givensub-goal in a path of attack.

In FIG. 7A, the costs of exploitation and remediation associated witheach attacker step are indicated using the format “X/Y”, where “X” isthe cost of exploitation and “Y” is the cost of remediation.

Based on the costs of exploitation shown in FIG. 7A, the calculation ofthe importance measures for the sub-goals are:

-   -   A. For sub-goal 1, multiply 1 (the number of paths in which it        is included) by 2 (the numerical value of cost of exploitation        of attacker step A, which is the only attacker step leading to        an occurrence of sub-goal 1 in a path) resulting in an        importance measure of 2 (1×2=2);    -   B. For sub-goal 2, multiply 1 (the number of paths in which it        is included) by 4 (the numerical value of cost of exploitation        of attacker step B, which is the only attacker step leading to        an occurrence of sub-goal 2 in a path) to get an importance        measure of 4 (1×4=4);    -   C. For sub-goal 3, multiply 3 (the number of paths in which it        is included) by 5 (the highest numerical value of all costs of        exploitation of attacker steps C and G, which lead to an        occurrence of sub-goal 3 in a path) to get an importance measure        of 15 (3×5=15);    -   D. For sub-goal 4, multiply 1 (the number of paths in which it        is included) by 6 (the numerical value of cost of exploitation        of attacker step E, which is the only attacker step leading to        an occurrence of sub-goal 4 in a path) to get an importance        measure of 6 (1×6=6);    -   E. For sub-goal 5, multiply 2 (the number of paths in which it        is included) by 8 (the highest numerical value of all costs of        exploitation of attacker steps F and I, which lead to an        occurrence of sub-goal 5 in a path) to get an importance measure        of 16 (2×8=16); and    -   F. For sub-goal 6, multiply 1 (the number of paths in which it        is included) by 3 (the numerical value of cost of exploitation        of attacker step H, which is the only attacker step leading to        an occurrence of sub-goal 6 in a path) to get an importance        measure of 3 (1×3=3).

Therefore, the order of importance of the sub-goals of FIG. 7A, usingthis second exemplary importance measure, from highest importance tolowest importance, is {sub-goal 5, sub-goal 3, sub-goal 4, sub-goal 2,sub-goal 6, sub-goal 1}.

A third example for a measure of importance of a given sub-goal is theaddition of twice the number of paths of attack in which the givensub-goal is included to the lowest cost of remediation of all attackersteps leading to an occurrence of the given sub-goal in a path ofattack. As in the second exemplary measure, the higher the numericalvalue of the importance measure, the more important is the sub-goal.

The reasoning behind using this measure is similar to the reasoningbehind that of the second exemplary measure, only in this case the“weighting factors” take into account the ease or difficulty with whicha sub-goal can be better protected (or made less vulnerable). In orderfor this exemplary measure to make sense, the definition of the cost ofremediation in this example should be such that the easier it is toblock an attacker step, the higher is the numerical value of the cost ofremediation of that attacker step. This implies that the “lowest cost ofremediation” used in the calculation is numerically the highest cost ofremediation of all attacker steps leading to an occurrence of the givensub-goal in a path of attack.

Based on the costs of remediation shown in FIG. 7A, the calculation ofthe importance measures for the sub-goals in the current example are:

-   -   A. For sub-goal 1, add two times 1 (the number of paths in which        it is included) to 6 (the numerical value of cost of remediation        of attacker step A, which is the only attacker step leading to        an occurrence of sub-goal 1 in a path) to get an importance        measure of 8 (2×1+max{6}=8);    -   B. For sub-goal 2, add two times 1 (the number of paths in which        it is included) to 9 (the numerical value of cost of remediation        of attacker step B, which is the only attacker step leading to        an occurrence of sub-goal 2 in a path) to get an importance        measure of 11 (2×1+max{9}=11);    -   C. For sub-goal 3, add two times 3 (the number of paths in which        it is included) to 3 (the highest numerical value of all costs        of remediation of attacker steps C and G, which lead to an        occurrence of sub-goal 3 in a path) to get an importance measure        of 9 (2×3+max{3, 2}=9);    -   D. For sub-goal 4, add two times 1 (the number of paths in which        it is included) to 5 (the numerical value of cost of remediation        of attacker step E, which is the only attacker step leading to        an occurrence of sub-goal 4 in a path) to get an importance        measure of 7 (2×1+max{5}=7);    -   E. For sub-goal 5, add two times 2 (the number of paths in which        it is included) to 6 (the highest numerical value of all costs        of remediation of attacker steps F and I, which lead to an        occurrence of sub-goal 5 in a path) to get an importance measure        of 10 (2×2+max{5, 6}=10); and    -   F. For sub-goal 6, add two times 1 (the number of paths in which        it is included) to 4 (the numerical value of cost of remediation        of attacker step H, which is the only attacker step leading to        an occurrence of sub-goal 6 in a path) to get an importance        measure of 6 (2×1+max{4}=6).

Therefore, the order of importance of the sub-goals of FIG. 7A, usingthis third exemplary importance measure, from highest importance tolowest importance, is {sub-goal 2, sub-goal 5, sub-goal 3, sub-goal 1,sub-goal 4, sub-goal 6}.

A fourth example for a measure of importance of a given sub-goal is theaddition of three times the number of paths of attack in which the givensub-goal is included to the sum of costs of remediation of all differentattacker steps leading to an occurrence of the given sub-goal in a pathof attack. As in the second and third exemplary measures, the higher thenumerical value of the importance measure, the more important is thesub-goal.

The reasoning behind using this measure is similar to the reasoningbehind that of the second and third exemplary measures, only in thiscase the “weighting factors” take into account the ease or difficultywith which a sub-goal can be fully protected (and not just made slightlybetter protected, as in the previous example). As in the previousexample, in order for this exemplary measure to make sense, thedefinition of the cost of remediation in this example should be suchthat the easier it is to block an attacker step, the higher is thenumerical value of the cost of remediation of that attacker step.

Based on the costs of remediation shown in FIG. 7A, the calculation ofthe importance measures for the sub-goals in the current example are:

-   -   A. For sub-goal 1, add three times 1 (the number of paths in        which it is included) to 6 (the numerical value of the cost of        remediation of attacker step A, which is the only attacker step        leading to an occurrence of sub-goal 1 in a path) to get an        importance measure of 9 (3×1+6=9);    -   B. For sub-goal 2, add three times 1 (the number of paths in        which it is included) to 9 (the numerical value of the cost of        remediation of attacker step B, which is the only attacker step        leading to an occurrence of sub-goal 2 in a path) to get an        importance measure of 12 (3×1+9=12);    -   C. For sub-goal 3, add three times 3 (the number of paths in        which it is included) to 5 (the sum of the numerical values of        the costs of remediation of attacker steps C and G that lead to        occurrences of sub-goal 3 in a path) to get an importance        measure of 14 (3×3+[3+2]=14);    -   D. For sub-goal 4, add three times 1 (the number of paths in        which it is included) to 5 (the numerical value of the cost of        remediation of attacker step E, which is the only attacker step        leading to an occurrence of sub-goal 3 in a path) to get an        importance measure of 8 (3×1+5=8);    -   E. For sub-goal 5, add three times 2 (the number of paths in        which it is included) to 11 (the sum of the numerical values of        the costs of remediation of attacker steps F and I that lead to        occurrences of sub-goal 5 in a path) to get an importance        measure of 17 (3×2+[5+6]=17); and    -   F. For sub-goal 6, add three times 1 (the number of paths in        which it is included) to 4 (the numerical value of the cost of        remediation of attacker step H, the only attacker step leading        to an occurrence of sub-goal 6 in a path) to get an importance        measure of 7 (3×1+4=7).

Therefore, the order of importance of the sub-goals of FIG. 7A, usingthis fourth exemplary importance measure, from highest importance tolowest importance, is {sub-goal 5, sub-goal 3, sub-goal 2, sub-goal 1,sub-goal 4, sub-goal 6}.

The importance measure calculations presented above are merely examples,and many other importance measures may be used when implementing thepresent invention, including importance measures based only on thenumber of paths of attack in which a sub-goal is included (as in thefirst example above), based on the number of paths of attack in which asub-goal is included as well as on other factors (such as costs ofexploitation and/or costs of remediation of attacker steps leading to asub-goal, as in the second, third and fourth examples above), or basedon other factors and considerations.

In the exemplary importance calculations discussed hereinabove usingFIG. 7A, sub-goal 3 was “credited” with three appearances in paths ofattack, even though two of those three appearances are preceded by thesame attacker step. In both the left and central paths of FIG. 7A,sub-goal 3 is reached by attacker step G. In some embodiments of theproposed invention, multiple occurrences of the same sub-goal that areincluded in multiple paths of attack and are reached by the sameattacker step, are counted as a single appearance for the purpose ofcalculating the importance measure of that sub-goal.

FIG. 7B presents the same paths of attack as FIG. 7A. However, in FIG.7B multiple appearances of a sub-goal reached by the same attacker stepare eliminated by merging such multiple appearances into one commonappearance. Specifically, comparison of FIGS. 7A and 7B demonstratesthat the appearances of sub-goal 3 in the central and left paths of FIG.7A (indicated by reference numerals 403 a in FIG. 7A), both of which arereached by step G, have been merged in FIG. 7B to a single appearance ofsub-goal 3 (indicated by reference numeral 403 b). Conversely, whenmultiple appearances of a sub-goal are reached by different attackersteps, those appearances are not merged into a single appearance in FIG.7B, and each one of the multiple appearances is separately counted. Forexample, the two appearances of sub-goal 5, which are reached by steps Fand I, have not been merged in FIG. 7B.

FIGS. 7A and 7B are completely equivalent, and provide two differentpresentation forms of the same information. However, FIG. 7B is moreconvenient when using an embodiment that counts multiple occurrences ofthe same sub-goal and attacker step combination as a single occurrenceof the sub-goal for the purpose of calculating the sub-goal importancemeasure.

The calculations of the importance measures for the sub-goals, accordingto the examples provided hereinabove, are repeated using an embodimentthat counts multiple occurrences of the same sub-goal that are reachedby the same attacker step as a single occurrence of the sub-goal (asshown in FIG. 7B).

For the first example, in which the importance measure is based only onnumber of appearances of a sub-goal in the graph of paths of attack, theresults are:

-   -   A. Sub-goal 1 has one appearance and consequently its importance        measure is 1;    -   B. Sub-goal 2 has one appearance and consequently its importance        measure is 1;    -   C. Sub-goal 3 has two appearances and consequently its        importance measure is 2;    -   D. Sub-goal 4 has one appearance and consequently its importance        measure is 1;    -   E. Sub-goal 5 has two appearances and consequently its        importance measure is 2; and    -   F. Sub-goal 6 has one appearance and consequently its importance        measure is 1.

For the second example, in which the importance measure is based on thenumber of appearances of a sub-goal in the graph of paths of attack aswell as on costs of exploitations of attacker steps leading to thesub-goal, the results are:

-   -   A. For sub-goal 1, multiply 1 (the number of appearances) by 2        (the numerical value of cost of exploitation of attacker step A,        which is the only attacker step leading to an occurrence of        sub-goal 1 in a path) to get an importance measure of 2 (1×2=2);    -   B. For sub-goal 2, multiply 1 (the number of appearances) by 4        (the numerical value of cost of exploitation of attacker step B,        which is the only attacker step leading to an occurrence of        sub-goal 2 in a path) to get an importance measure of 4 (1×4=4);    -   C. For sub-goal 3, multiply 2 (the number of appearances) by 5        (the highest numerical value of all costs of exploitation of        attacker steps C and G which lead to an occurrence of sub-goal 3        in a path) to get an importance measure of 10 (2×5=10);    -   D. For sub-goal 4, multiply 1 (the number of appearances) by 6        (the numerical value of cost of exploitation of attacker step E,        which is the only attacker step leading to an occurrence of        sub-goal 4 in a path) to get an importance measure of 6 (1×6=6);    -   E. For sub-goal 5, multiply 2 (the number of appearances) by 8        (the highest numerical value of all costs of exploitation of        attacker steps F and I which lead to an occurrence of sub-goal 5        in a path) to get an importance measure of 16 (2×8=16); and    -   F. For sub-goal 6, multiply 1 (the number of appearances) by 3        (the numerical value of cost of exploitation of attacker step H,        which is the only attacker step leading to an occurrence of        sub-goal 6 in a path) to get an importance measure of 3 (1×3=3).

For the third example, in which the importance measure is based on thenumber of appearances of a sub-goal in the graph of paths of attack aswell as on costs of remediation of attacker steps leading to thesub-goal, the results are:

-   -   A. For sub-goal 1, add two times 1 (the number of appearances)        to 6 (the numerical value of cost of remediation of attacker        step A, which is the only attacker step leading to an occurrence        of sub-goal 1 in a path) to get an importance measure of 8        (2×1+max{6}=8);    -   B. For sub-goal 2, add two times 1 (the number of appearances)        to 9 (the numerical value of cost of remediation of attacker        step B, which is the only attacker step leading to an occurrence        of sub-goal 2 in a path) to get an importance measure of 11        (2×1+max{9}=11);    -   C. For sub-goal 3, add two times 2 (the number of appearances)        to 3 (the highest numerical value of all costs of remediation of        attacker steps C and G which lead to an occurrence of sub-goal 3        in a path) to get an importance measure of 7 (2×2+max{3, 2}=7);    -   D. For sub-goal 4, add two times 1 (the number of appearances)        to 5 (the numerical value of cost of remediation of attacker        step E, which is the only attacker step leading to an occurrence        of sub-goal 4 in a path) to get an importance measure of 7        (2×1+max{5}=7);    -   E. For sub-goal 5, add two times 2 (the number of appearances)        to 6 (the highest numerical value of all costs of remediation of        attacker steps F and I which lead to an occurrence of sub-goal 5        in a path) to get an importance measure of 10 (2×2+max {5,        6}=10); and    -   F. For sub-goal 6, add two times 1 (the number of appearances)        to 4 (the numerical value of cost of remediation of attacker        step H, which is the only attacker step leading to an occurrence        of sub-goal 6 in a path) to get an importance measure of 6        (2×1+max{4}=6).

For the fourth example, in which the importance measure is also based onthe number of appearances as well as costs of remediation (as in thethird example), the results are:

-   -   A. For sub-goal 1, add three times 1 (the number of appearances)        to 6 (the numerical value of the cost of remediation of attacker        step A, the only attacker step leading to an occurrence of        sub-goal 1 in a path) to get an importance measure of 9        (3×1+6=9);    -   B. For sub-goal 2, add three times 1 (the number of appearances)        to 9 (the numerical value of the cost of remediation of attacker        step B, the only attacker step leading to an occurrence of        sub-goal 2 in a path) to get an importance measure of 12        (3×1+9=12);    -   C. For sub-goal 3, add three times 2 (the number of appearances)        to 5 (the sum of the numerical values of the costs of        remediation of attacker steps C and G that lead to occurrences        of sub-goal 3 in a path) to get an importance measure of 11        (3×2+[2+3]=11);    -   D. For sub-goal 4, add three times 1 (the number of appearances)        to 5 (the numerical value of the cost of remediation of attacker        step E, which is the only attacker step leading to an occurrence        of sub-goal 4 in a path) to get an importance measure of 8        (3×1+5=8);    -   E. For sub-goal 5, add three times 2 (the number of appearances)        to 11 (the sum of the numerical values of the costs of        remediation of attacker steps F and I that lead to occurrences        of sub-goal 5 in a path) to get an importance measure of 17        (3×2+[5+6]=17); and    -   F. For sub-goal 6, add three times 1 (the number of appearances)        to 4 (the numerical value of the cost of remediation of attacker        step H, which is the only attacker step leading to an occurrence        of sub-goal 6 in a path) to get an importance measure of 7        (3×1+4=7).

FIG. 7C is another form of representation of the paths of attack ofFIGS. 7A and 7B, and is completely equivalent to each of them. In FIG.7C each sub-goal appears only once. Even if a sub-goal is reachable bytwo different attacker steps, it still appears only once in the graph ofpaths of attack. For example, sub-goal 5, which is reachable by bothstep F and step I, appears only once in the graph, unlike in FIG. 7Bwhere it appears twice. FIG. 7C may be used instead of FIG. 7B whencalculating importance measures for an embodiment that counts multipleoccurrences of the same sub-goal and attacker step combination as asingle occurrence of the sub-goal. However, when using FIG. 7C for thispurpose, care must be taken to consider a sub-goal reachable by multipledifferent attacker steps as having multiple occurrences.

As discussed hereinabove with respect to FIGS. 3A to 6, in some cases, asub-goal included in a path of attack may be the compromising of a givennetwork node, such as, for example, compromising the CFO's computer orcompromising the inventory database server. However, sub-goals are notlimited to compromising network nodes, and any achievement that may beuseful to an attacker in any way may serve as a sub-goal in a path ofattack. For example, a sub-goal may be the obtaining of certainresources (see for example at sub-goals 310 and 406 in FIG. 6), such asa passwords file of the accounting department or the credentials (e.g.username and password) of a given user.

Given one's ability to calculate an importance measure of any sub-goalincluded in at least one path of attack, in accordance with the presentinvention there is provided a method of recommending the best sub-goalsto be protected by remediation actions, which method includes thefollowing operative steps:

-   1. Initialize an ordered list of sub-goals to be protected to be an    empty list.-   2. Define a halting condition for stopping operation of the method.    The halting condition is a condition applied to the ordered list of    sub-goals to be protected. For example, the halting condition may be    that the ordered list includes three members, or that the cumulative    sum of the costs of remediation of all attacker steps leading to    members in the ordered list exceeds a given threshold.-   3. For each sub-goal included in at least one path of attack,    calculate its corresponding importance measure.-   4. Determine the sub-goal that has the optimal importance measure.    The optimal importance measure may be the highest numerical value of    importance measure, when the importance measure generates a high    numerical value for a high importance sub-goal, as in the examples    provided above, or it may be the lowest numerical value of    importance measure, when the importance measure generates a high    numerical value for a low importance sub-goal. In the case of a tie,    in which multiple sub-goals have equal corresponding importance    measures, employ some tie-breaking mechanism to determine the    optimal sub-goal. The tie-breaking mechanism may depend on factors    such as costs of remediation, costs of exploitation and/or    probabilities of success of the attacker steps leading to the    sub-goals having the same importance measure. Alternatively, the    tie-breaking mechanism may be a random selection of one of the    sub-goals having the same importance measure.-   5. Insert the sub-goal determined to have the optimal importance    measure to be the next member (last member) in the ordered list of    sub-goals to be protected.-   6. Modify the representation of the networked system vulnerabilities    (e.g. the graph representing the paths of attack or the list    representing the paths of attack), by removing from it all    occurrences of the last sub-goal inserted into the ordered list. Any    path of attack that becomes “broken” and unavailable to the attacker    by the removal of the sub-goal, is entirely removed from the    representation.-   7. Evaluate the halting condition for the ordered list.-   8. If the halting condition is not satisfied and there is at least    one path of attack left in the modified representation, return to    step 3, using the modified representation as the current    representation of the networked system vulnerabilities. Otherwise,    if the halting condition is satisfied or there are no more paths of    attack left in the modified representation, terminate the    computation and provide the current ordered list as the output of    the method.

The inventive method described above provides as its output an orderedlist of sub-goals whose protection, for example by remediation actions,would improve the security of the networked system. Alternatively, theoutput may be an unordered list of sub-goals to be protected. Theadvantage of having the list of attacker steps in a specific order isthat, if the cost required for protecting all of the recommendedsub-goals is greater than the budget available for improving thesecurity of the networked system, the relative importance of protectingthe sub-goals in the list is known, and the best use of the existingresources for protecting the system becomes clear to the system owner oradministrator.

As an example, the inventive method described herein is applied to theexample illustrated in FIG. 7A. For the purpose of present example, theimportance measure of a sub-goal is defined as in the first examplepresented above (i.e, equal to the number of paths of attack includingthe sub-goal). The embodiment used in this example counts everyoccurrence of a sub-goal in a path of attack, even when the sameattacker step leads to the sub-goal in multiple paths of attack.

In this exemplary setting of FIG. 7A, the method described hereinaboveoperates as follows:

-   1. The ordered list of sub-goals to be protected is initialized to {    } (the empty list).-   2. The halting condition is set to be “having two members in the    ordered list”.-   3. Calculating the importance measure for each sub-goal (see the    calculations in the first example above):    -   a. Sub-goal 1—importance measure of 1;    -   b. Sub-goal 2—importance measure of 1;    -   c. Sub-goal 3—importance measure of 3;    -   d. Sub-goal 4—importance measure of 1;    -   e. Sub-goal 5—importance measure of 2;    -   f. Sub-goal 6—importance measure of 1;-   4. The sub-goal having the highest importance is sub-goal 3.-   5. The ordered list of sub-goals to be protected becomes {sub-goal    3}.-   6. The representation of the networked system vulnerabilities (i.e.    the graph of FIG. 7A) is modified by removing all occurrences of    sub-goal 3. As this breaks all three paths of attack, all of them    are completely removed from the representation.-   7. The halting condition is evaluated and found to be not satisfied,    since there is only a single sub-goal in the ordered list.-   8. As no path of attack remains in the graph, the inventive method    terminates with the result being the list {sub-goal 3}.

In this example, the remediation recommendation that is provided is toprotect only sub-goal 3. In should be noted, that protecting a givensub-goal means blocking each and every attacker step that leads to anoccurrence of the given sub-goal in a path of attack. In the example ofFIG. 7A, a recommendation to protect sub-goal 3 translates into arecommendation to block attacker steps C and G.

In the present example, even though the halting condition requires thatthe ordered list have two members (i.e. the question we asked was “whatare the two best sub-goals to be protected?”), the result produced bythe inventive method is a recommendation to protect a single sub-goal.This is due to the fact that once sub-goal 3 is protected, there is noadded benefit in protecting another sub-goal, as the protection ofsub-goal 3 already brings the networked system to be fully protected.Obviously, this is not always the case, and in most real-world cases theinventive method will generate multiple recommendations of sub-goals tobe protected, ordered from most important to least important.

As explained above, the identity of the specific sub-goals to beprotected, depends very much on the selected importance measure.

In order to demonstrate this, in a second exemplary implementation, themethod of the present invention is applied to the example illustrated inFIG. 7A while using the third exemplary importance measure discussedhereinabove (the importance measure of a sub-goal is the addition oftwice the number of paths of attack in which the sub-goal is included tothe lowest cost of remediation of all attacker steps leading to anoccurrence of the sub-goal in a path of attack). The embodiment used inthis example counts every occurrence of a sub-goal in a path of attack,even when the same attacker step leads to the sub-goal in multiple pathsof attack.

In this second exemplary setting of FIG. 7A, the method describedhereinabove operates as follows:

-   1. The ordered list of sub-goals to be protected is initialized to {    } (the empty list).-   2. The halting condition is set to be “having two members in the    ordered list”.-   3. Calculating the importance measure for each sub-goal based on the    graph shown in FIG. 7A (see the calculations in the third example    above):    -   a. Sub-goal 1—importance measure of 8 (2×1+max{6}=8);    -   b. Sub-goal 2—importance measure of 11 (2×1+max{9}=11);    -   c. Sub-goal 3—importance measure of 9 (2×3+max{3, 2}=9);    -   d. Sub-goal 4—importance measure of 7 (2×1+max{5}=7);    -   e. Sub-goal 5—importance measure of 10 (2×2+max{5, 6}=10); and    -   f. Sub-goal 6—importance measure of 6 (2×1+max{4}=6).-   4. The sub-goal having the highest importance is sub-goal 2.-   5. The ordered list of sub-goals to be protected becomes {sub-goal    2}.-   6. The representation of the networked system vulnerabilities (i.e.    the graph of FIG. 7A) is modified by removing all occurrences of    sub-goal 2. This breaks only the rightmost path of attack, which is    completely removed from the representation, thus forming the    representation shown in FIG. 8.-   7. The halting condition is evaluated and found to be not satisfied,    since there is only a single sub-goal in the ordered list.-   8. As two paths of attack remains in the graph (the central path of    attack and the leftmost path of attack), the inventive method    carries out another iteration, starting at step 3.-   3a. Recalculating the importance measure for each sub-goal based on    the modified representation illustrated in FIG. 8:    -   a. Sub-goal 1—has been removed from the representation;    -   b. Sub-goal 2—has been removed from the representation;    -   c. Sub-goal 3—importance measure of 6 (2×2+max{2, 2}=6);    -   d. Sub-goal 4—importance measure of 7 (2×1+max{5}=7);    -   e. Sub-goal 5—importance measure of 10 (2×2+max{5, 6}=10); and    -   f. Sub-goal 6—importance measure of 6 (2×1+max{4}=6).-   4a. The sub-goal having the highest importance is sub-goal 5.-   5a. The ordered list of sub-goals to be protected becomes {sub-goal    2, sub-goal 5}.-   6a. The representation of the networked system vulnerabilities (i.e.    the graph of FIG. 8) is modified by removing all occurrences of    sub-goal 5. This breaks the remaining two paths of attack, which are    both completely removed from the representation.-   7a. The halting condition is evaluated and found to be satisfied,    since there are two sub-goal in the ordered list.-   8a. As the halting condition is satisfied, and no path of attack    remains in the graph, the inventive method terminates with the    result being the list {sub-goal 2, sub-goal 5}.

In this example, the remediation recommendation that is provided is toprotect sub-goals 2 and 5. In the example of FIG. 7A, a recommendationto protect sub-goal 2 translates into a recommendation to block attackerstep B, and a recommendation to protect sub-goal 5 translates into arecommendation to block attacker steps F and I.

As already explained above, in FIG. 7A the three branches, or paths ofattack, are drawn separately, even though they share a sub-goal (i.e.shared sub-goal 3 is drawn three times). Alternatively, the threebranches may be drawn using two occurrences of the shared sub-goal (asin FIG. 7B) or using only a single occurrence of the shared sub-goal (asin FIG. 7C). All these forms are equivalent, and the inventive method isapplicable for all of them.

Additionally, in the example of FIG. 7C the two left-most paths ofattack merge into a single path. This occurs when two paths share acommon sub-goal (in this case sub-goal 5). A vulnerabilities graphincluding multiple branches merging into a common ending is equivalentto a vulnerabilities graph containing separately drawn branchesduplicating the common portion of the merging branches. Therefore, theinventive method is equally applicable to cases in which sub-goals areshared between different branches, for example when multiple branchesmerge into a common ending.

Similarly, a branch or path of attack may split into multiple differentcontinuations. This happens when there are multiple attacker steps thatmay be used by the attacker to continue his attack after reaching acertain sub-goal. A vulnerabilities graph drawn this way is alsoequivalent to a vulnerabilities graph containing separately drawnbranches, duplicating the common initial portion of the splitting branchin each one of the separately drawn branches. Therefore, the inventivemethod is equally applicable to cases in which a common beginning splitsinto multiple branches.

It should be emphasized that applying the inventive method disclosedherein does not necessarily require the use of a graph. As discussedhereinabove, a graph is just one form of representation of the paths ofattack discovered in a networked system, and there are other,equivalent, forms of representation. The inventive method may equally beapplied while using a list or some other data structure for representingthe paths of attack of a networked system.

When the inventive method is used for identifying a single optimalsub-goal to be protected, and there is no interest in determiningmultiple sub-goals to be protected, the method may be somewhatsimplified. For this special case the method reduces to:

-   -   1. For each sub-goal included in at least one path of attack,        calculate its corresponding importance measure.    -   2. Determine the sub-goal that has the optimal importance        measure. The optimal importance measure may be the highest        numerical value of importance measure, when the importance        measure generates a high numerical value for a high importance        sub-goal, as in the examples provided above, or it may be the        lowest numerical value of importance measure, when the        importance measure generates a high numerical value for a low        importance sub-goal. In the case of a tie, in which multiple        sub-goals have equal corresponding importance measures, employ        some tie-breaking mechanism to determine the optimal sub-goal.        The tie-breaking mechanism may depend on factors such as costs        of remediation, costs of exploitation and/or probabilities of        success of the attacker steps leading to the sub-goals having        the same importance measure. Alternatively, the tie-breaking        mechanism may be a random selection of one of the sub-goals        having the same importance measure.    -   3. The result of the inventive method is the sub-goal having the        optimal importance measure.

As can easily be seen, applying the above single-sub-goal version of theinventive method to the example of FIG. 7A using the first exemplaryimportance measure (number of paths of attack in which a sub-goalappears) results in a recommendation to protect sub-goal 3, the sameresult shown above to be generated by the multiple-sub-goals version ofthe inventive method using the same importance measure.

Reference is now made to FIGS. 9A and 9B, which illustrate paths ofattack discovered for a tested networked system, and with respect towhich the method of the present invention will also be described.

FIGS. 9A and 9B initiate at a Starting state 500 and include attackersteps 502 (Use method A), 504 (Use method B), 506 (Use method C), 508(Use method D), 510 a (Use method E1), 510 b (Use method E2), 510 c (Usemethod E3), 512 a (Use method F1), 512 b (Use method F2), and 512 c (Usemethod F3). FIGS. 9A and 9B also include sub-goal 1 indicated byreference numeral 532, sub-goal 2 indicated by reference numeral 534,sub-goal 3 indicated by reference numeral 536, sub-goal 4 indicated byreference numeral 538, sub-goal 5 indicated by reference numeral 540,sub-goal 6 indicated by reference numeral 542, and final sub-goal 544.

FIGS. 9A and 9B use a representation mode in which paths of attack thatshare common attacker steps and common sub-goals are drawn as a commonbranch that splits into multiple sub-branches. FIGS. 9A and 9B indicatethat the networked system has four paths of attack:

-   A. {Starting state, Use method A, Sub-Goal 1, Use method D, Final    Sub-Goal}-   B. {Starting state, Use method B, Sub-Goal 2, Use method C, Sub-Goal    3, Use method E1, Sub-Goal 4, Use method F1, Final Sub-Goal}-   C. {Starting state, Use method B, Sub-Goal 2, Use method C, Sub-Goal    3, Use method E2, Sub-Goal 5, Use method F2, Final Sub-Goal}-   D. {Starting state, Use method B, Sub-Goal 2, Use method C, Sub-Goal    3, Use method E3, Sub-Goal 6, Use method F3, Final Sub-Goal}

The three attacker steps 510 a, 510 b, and 510 c (Use method E1, Usemethod E2, and Use method E3, respectively), are similar to each otherand share the same cost of exploitation and cost of remediation. Forexample, attacker step 510 a (Use method E1) may be the attacker step“using common technique E for obtaining an ability to remotely executearbitrary code in network node X”, attacker step 510 b (Use method E2)may be the attacker step “using common technique E for obtaining anability to remotely execute arbitrary code in network node Y”, andattacker step 510 c (Use method E3) may be the attacker step “usingcommon technique E for obtaining an ability to remotely executearbitrary code in network node Z”.

Similarly, the three attacker steps 512 a, 512 b, and 512 c (Use methodF1, Use method F2, and Use method F3, respectively), are similar to eachother and share the same cost of exploitation and cost of remediation.In the above example, attacker step 512 a (Use method F1) may be theattacker step “using common technique F and an ability to remotelyexecute arbitrary code in network node X, obtain the final sub-goal”,attacker step 512 b (Use method F2) may be the attacker step “usingcommon technique F and an ability to remotely execute arbitrary code innetwork node Y, obtain the final sub-goal”, and attacker step 512 c (Usemethod F3) may be the attacker step “using common technique F and anability to remotely execute arbitrary code in network node Z, obtain thefinal sub-goal”.

In the above example the three sub-goals 538, 540, and 542 betweenattacker steps 510 a, 510 b, and 510 c, and attacker steps 512 a, 512 b,and 512 c, respectively, are also similar to each other. Sub-goal 4(indicated by reference numeral 538) may be “an ability to remotelyexecute arbitrary code in network node X”, sub-goal 5 (indicated byreference numeral 540) may be “an ability to remotely execute arbitrarycode in network node Y”, and sub-goal 6 (indicated by reference numeral542) may be “an ability to remotely execute arbitrary code in networknode Z”.

Such cases of paths of attack that include similar sub-goals that followsimilar attacker steps and are followed by similar attacker steps, arequite common. For example, common technique E may be a technique thatcompromises a network node of choice, and common technique F may be atechnique that achieves the next sub-goal using the node compromised byuse of technique E. Network nodes X, Y and Z may be three nodes thatshare a common broadcast domain with a network node W that is associatedwith the next sub-goal, which, in this example, is the network node thatis associated with the final goal 544 of the attacker. In order toachieve a desired sub-goal in node W, an attacker may compromise any oneof the three nodes located in the same broadcast domain as node W (usingtechnique E), and then remotely execute on the compromised nodemalicious code that achieves the final goal in node W (using techniqueF). From the attacker's point of view, each of the three nodes X, Y andZ may be used, thus creating three paths of attack, that arefunctionally equivalent but differ in the network node compromised as asub-goal on the way to achieving the desired next sub-goal.

As in previous Figures, in FIGS. 9A and 9B the costs associated witheach attacker step are indicated using the format “X/Y”, where “X” isthe cost of exploitation and “Y” is the cost of remediation.

The importance measure used for the example relating to FIGS. 9A and 9Bis the second exemplary importance measure described above—themultiplication of the number of paths of attack in which the givensub-goal is included by the lowest cost of exploitation of all attackersteps leading to an occurrence of the given sub-goal in a path ofattack. As noted above, this importance measure is defined such that alower numerical measure indicates a less important sub-goal, and ahigher numerical measure indicates a more important sub-goal.Additionally, as noted above, when portions of multiple paths of attackare merged in the representation because they include identical attackersteps leading to identical sub-goals, those sub-goals are counted asappearing in each of the multiple paths of attack, even though in thegraphic representation those sub-goals may appear only once.

Turning now to use of the inventive method, in a first stage, thesingle-sub-goal version of the inventive method is applied to theexemplary graph of FIGS. 9A and 9B.

As a first step, sub-goal importance levels are computed for each of thesub-goals included in any of the paths of attack:

The importance level calculations applied to FIGS. 9A and 9B result in:

-   -   a. Sub-goal 1—importance of: 1×2=2 (one appearance of sub-goal        1, attacker step 502 leading to sub-goal 1 has a cost of        exploitation of 2);    -   b. Sub-goal 2—importance of: 3×3=9 (three [merged] appearances        of sub-goal 2, step 504 leading to sub-goal 2 has a cost of        exploitation of 3);    -   c. Sub-goal 3—importance of: 3×3=9 (three [merged] appearances        of sub-goal 3, step 506 leading to sub-goal 3 has a cost of        exploitation of 3);    -   d. Sub-goal 4—importance of: 1×4=4 (one appearance of sub-goal        4, step 510 a leading to sub-goal 4 has a cost of exploitation        of 4);    -   e. Sub-goal 5—importance of: 1×4=4 (one appearance of sub-goal        5, step 510 b leading to sub-goal 5 has a cost of exploitation        of 4); and    -   f. Sub-goal 6—importance of: 1×4=4 (one appearance of sub-goal        6, step 510 c leading to sub-goal 6 has a cost of exploitation        of 4).

Next, the optimal sub-goal is determined by finding the numericallyhighest value of the sub-goal importance measures listed above. Thehighest sub-goal importance measure is 9, which is shared by sub-goals 2and 3. Because two sub-goals are tied for the highest importancemeasure, a tie-breaking mechanism is used. In the present example, thetie-breaking rule selects the sub-goal having the lowest cost ofremediation when blocking all attacker steps leading to that sub-goal.As attacker step 506 (“Use method C”), which is the only attacker stepleading to sub-goal 3, has a lower cost of remediation than attackerstep 504 (“Use method B”), which is the only attacker step leading tosub-goal 2, (3 vs. 9), the method generates a recommendation to protectsub-goal 3, which translates to blocking attacker step 506.

In a second stage, the multiple-sub-goals version of the inventivemethod is applied to the example of FIGS. 9A and 9B. The exemplaryhalting condition to be used is having an ordered list of sub-goals thatcontains three sub-goals.

The first iteration of the multiple-sub-goals version of the methodoperates exactly as the single-sub-goals version and inserts sub-goal 3into the ordered list of sub-goals.

In the second iteration, the process is repeated for a group of paths ofattack that includes only path A, since paths B, C and D are “broken” byprotecting sub-goal 3 and are thus removed from consideration followingthe first iteration.

As sub-goal 1 is the only sub-goal in path A, it is the sub-goalselected to be protected in the second iteration. As such, following thesecond iteration of the inventive method, the ordered list of sub-goalsis now {sub-goal 3, sub-goal 1}.

The halting condition is now checked, but is not satisfied, as the listincludes only two members and not three. However, no paths of attackremain in the graph at this stage, causing the inventive method toterminate with a recommendation for protecting (i) sub-goal 3, and (ii)sub-goal 1, in that order of priority.

Reference is now made to FIG. 10A, which is a block diagram of a system600 for providing a recommendation for improving the security of anetwork system against attackers according to an embodiment of a firstaspect of the present invention.

As seen in FIG. 10A, the system 600 includes apenetration-testing-campaign module 610, including one or morepenetration-testing-campaign processors 612 and apenetration-testing-campaign memory 614, such as a non-transitorycomputer readable storage medium, having stored thereon instructions tobe executed by the one or more penetration-testing-campaign processors612. The memory 614 has stored instructions to carry out one or morepenetration tests of a networked system.

System 600 further includes a reconnaissance module 616 adapted to carryout a reconnaissance function, an attack module 618 adapted to carry outan attack function, and optionally a cleanup module 619 adapted to carryout a cleanup function as described hereinabove. Thepenetration-testing-campaign module 610 may include at least part of thereconnaissance module 616 carrying out the reconnaissance function code,the attack module 618 carrying out the attack function code, andoptionally the cleanup module 619 carrying out the cleanup functioncode.

System 600 further includes a sub-goal-selection module 620, includingone or more sub-goal-selection processors 622 and a sub-goal-selectionmemory 624, such as a non-transitory computer readable storage medium,having stored thereon instructions to be executed by the one or moresub-goal-selection processors 622. The instructions include instructionsthat, when carried out in the correct order, select the single optimalsub-goal to be protected in order to improve the security of a testednetworked system.

Turning additionally to FIG. 10B, which is a schematic block diagram ofsub-goal-selection module 620, it is seen that memory 624 includesmultiple instructions to be executed by the processor(s) 622, including:

First instructions 625 to receive, from penetration-testing-campaignmodule 610, results of the one or more penetration tests of thenetworked system carried out by the penetration-testing-campaign module;

Second instructions 626 to determine, based on the received results,multiple paths of attack available to the attackers. As described indetail hereinabove, each such path of attack is an ordered sequence ofone or more attacker steps and one or more sub-goals.

Third instructions 627 to assign a calculated importance score, orimportance measure, to each of multiple sub-goals. Each sub-goal of themultiple sub-goals is included in at least one of the multiple paths ofattack determined by execution of second instructions 626. Additionally,for at least one given sub-goal of the multiple sub-goals, theimportance score or measure assigned to the given sub-goal is based onthe number of paths of attack which include the sub-goal, as in the fourexemplary importance measures described hereinabove; and

Fourth instructions 628 to select one sub-goal included in at least oneof the multiple paths of attack determined by execution of secondinstructions 626, where the one sub-goal is selected based on theimportance score assigned to at least one of the multiple sub-goals,and/or to that one sub-goal, during execution of the third instructions627.

Returning to FIG. 10A, it is seen that system 600 further includes areporting module 640, functionally associated with a user interface 650and with sub-goal-selection module 620. Reporting module 640 includesone or more reporting processors 642, and a reporting memory 644, suchas a non-transitory computer readable storage medium, having storedthereon instructions to be executed by the one or more reportingprocessors 642. The reporting memory 644 has stored:

instructions to receive from sub-goal-selection module 620 anidentification of the selected sub-goal; and

instructions to provide a recommendation to protect the selectedsub-goal, thereby to improve the security of the networked system. Theinstructions to provide the recommendation include at least one of:

-   -   (i) instructions to cause a display device (e.g. of a user        interface 650 or another display device, which may be located        remotely to the reporting module 640) to display information        about the recommendation;    -   (ii) instructions to store the information about the        recommendation in a file; and    -   (iii) instructions to electronically transmit the information        about the recommendation, for example using a transceiver 652        functionally associated with the reporting module 640.

In some embodiments, the penetration-testing-campaign memory 614,sub-goal-selection memory 624, and reporting memory 644 are each adedicated, and separate, memory component or storage medium. In otherembodiments, at least two of the penetration-testing-campaign memory614, the sub-goal-selection memory 624, and the reporting memory 644 maybe part of the same memory component or storage medium.

In some embodiments, the penetration-testing-campaign processor(s) 612,sub-goal-selection processor(s) 622, and reporting processor(s) 642 areeach dedicated, and separate, processors. In other embodiments, at leasttwo of the penetration-testing-campaign processor(s) 612,sub-goal-selection processor(s) 622, and reporting processor(s) 642share at least one common processor.

FIG. 11 is a flow chart of a method for providing a recommendation forimproving the security of a network system against attackers accordingto an embodiment of the first aspect of the invention. The method ofFIG. 11 is described herein as being carried out using system 600 ofFIG. 10A, but may be carried out using any suitable system.

In step S700 shown in FIG. 11, a penetration testing system, such aspenetration testing system 600, carries out one or more tests of thenetworked system, for example using testing methods known in the art.

Based on the results of the tests received by sub-goal-selection module620, for example by execution of first instructions 625 ofsub-goal-selection module 620, multiple paths of attack which areavailable to attackers are determined at step S702, for example byexecution of second instructions 626 of sub-goal-selection module 620.Each such path of attack is an ordered sequence of one or more attackersteps and one or more sub-goals.

In some embodiments, each path of attack determined in step S702 startsat a starting sub-goal that is assumed to be achievable by the attacker,and ends at a final sub-goal which is assumed to be a goal of theattacker in at least one of the tests carried out at step S700. Forexample, such paths of attack are illustrated in FIGS. 3A-9Bhereinabove.

In some embodiments, in each path of attack determined in step S702: (A)each attacker step in the path of attack is preceded by a sub-goal andfollowed by a sub-goal in the path of attack, (B) each sub-goal in thepath of attack, except for a starting sub-goal of the path of attack, ispreceded by an attacker step in the given path of attack, and (C) eachsub-goal in the path of attack, except for a final sub-goal of the pathof attack, is followed by an attacker step in the path of attack.

In some embodiments, at an optional step S704 a, the paths of attackdetermined at step S702 are represented by a graph, for example bysub-goal-selection module 620. Each determined path of attackcorresponds to a path in the graph.

In some embodiments, for each given path in the graph corresponding to aspecific path of attack available to the attackers, all sub-goals andall attacker steps included in the specific path of attack arerepresented as graph nodes included in the given path in said graph.Such a representation is illustrated, for example, in FIGS. 3A, 4, 5, 6,9A and 9B.

In some embodiments, for each given path in the graph corresponding to aspecific path of attack available to the attackers, all sub-goalsincluded in the specific path of attack are represented as graph nodesincluded in the given path in the graph and all attacker steps includedin the specific path of attack are represented as graph edges includedin the given path in the graph. Such a representation is illustrated,for example, in FIGS. 3B, and 7A to 8.

In some embodiments, at an optional step S704 b, the paths of attackdetermined at step S702 are represented by a list, for example bysub-goal-selection module 620. Each given path of attack corresponds toan item in the list that includes all sub-goals and all attacker stepsincluded in the given path of attack. Such a representation isillustrated, for example, in FIG. 3C.

A calculated importance score, or importance measure, is assigned toeach of multiple sub-goals at step S706, for example by execution ofthird instructions 627 of sub-goal-selection module 620. Each sub-goalof the multiple sub-goals to which an importance score is assigned, isincluded in at least one of the multiple paths of attack determined atstep S702. The importance score assigned to at least one given sub-goalof the multiple sub-goals is based on a number of determined paths ofattack which include that given sub-goal.

In some embodiments, the importance score assigned, at step S706, toeach given sub-goal of the multiple sub-goals, is based on the number ofdetermined paths of attack which include that given sub-goal.

In some embodiments, the importance score assigned, at step S706, to atleast one of the multiple sub-goals, is based only on the number ofdetermined paths of attack which include that sub-goal.

In some embodiments, the importance score assigned, at step S706, toeach given sub-goal of the multiple sub-goals, is based only on thenumber of determined paths of attack which include the given sub-goal.

In some embodiments, the importance score assigned, at step S706, to atleast one of the multiple sub-goals, is equal to the number ofdetermined paths of attack which include that sub-goal.

In some embodiments, the importance score assigned, at step S706, to atleast one of the multiple sub-goals, is based on (i) the number of pathsof attack of the determined multiple paths of attack which include thegiven sub-goal, and (ii) at least one member selected from the groupconsisting of: a cost of remediation of the given sub-goal, a cost ofexploitation of an attacker step leading to the given sub-goal, a costof remediation of an attacker step leading to the given sub-goal, and aprobability of success of an attacker step leading to the givensub-goal.

In some embodiments, the importance scores assigned at step S706 aresuch that a higher numerical value of the importance score for aspecific sub-goal is indicative of a higher degree of importanceassociated with the specific sub-goal.

In some other embodiments, the importance scores assigned at step S706are such that a lower numerical value of the importance score for aspecific sub-goal is indicative of a higher degree of importanceassociated with the specific sub-goal.

At step S710, a single specific sub-goal included in at least one of themultiple paths of attack determined at step S702, is selected, forexample by execution of fourth instructions 628 of sub-goal-selectionmodule 620. The selection of the specific sub-goal is based on theimportance score assigned to at least one of the multiple sub-goals atstep S706.

In some embodiments, the selected sub-goal is one whose assignedimportance score meets a predefined criterion.

In some embodiments, the selected sub-goal is one whose assignedimportance score has the highest numerical value of all importancescores assigned to sub-goals included in at least one of the paths ofattack determined at step S702. In other embodiments, the selectedsub-goal is one whose assigned importance score has the lowest numericalvalue of all importance scores assigned to sub-goals included in atleast one of the determined paths of attack.

In some embodiments, it may happen that multiple sub-goals have assignedthereto a common importance score, which is an importance scoreindicative of the highest degree of importance of all the importancescores assigned to the multiple sub-goals at step S706. In suchembodiments, a tie-breaking rule is used to select one of those multiplesub-goals as the single specific sub-goal.

In some such embodiments, the tie-breaking rule depends on at least oneof:

-   -   a cost of remediation of at least one of the multiple sub goals        having assigned the common importance score;    -   a cost of exploitation of at least one attacker step leading to        at least one of the multiple sub-goals having assigned the        common importance score;    -   a cost of remediation of at least one attacker step leading to        at least one of the multiple sub-goals having assigned the        common importance score; and    -   a probability of success of at least one attacker step leading        to at least one of the multiple sub-goals having assigned the        common importance score.

In some other such embodiments, the tie-breaking rule includes randomlyselecting one of the multiple sub-goals having assigned the commonimportance score as the single specific sub-goal.

At step S712, a recommendation is provided to protect the selectedsingle specific sub-goal, thereby to improve the security of thenetworked system. For example, the recommendation may be provided byreporting module 640. The recommendation may be provided by any one ormore of: (i) causing a display device to display information about therecommendation, (ii) recording the information about the recommendationin a file, and (iii) electronically transmitting the information aboutthe recommendation, for example to a remote location.

Reference is now made to FIG. 12A, which is a block diagram of a system800 for providing a recommendation for improving the security of anetwork system against attackers according to an embodiment of a secondaspect of the present invention.

As seen in FIG. 12A, the system 800 includes apenetration-testing-campaign module 810, including one or morepenetration-testing-campaign processors 812 and apenetration-testing-campaign memory 814, such as a non-transitorycomputer readable storage medium, having stored thereon instructions tobe executed by the one or more penetration-testing-campaign processors812. The memory 814 has stored instructions to carry out one or morepenetration tests of a networked system.

System 800 further includes a reconnaissance module 816 adapted to carryout a reconnaissance function, an attack module 818 adapted to carry outan attack function, and optionally a cleanup module 819 adapted to carryout a cleanup function as described hereinabove. Thepenetration-testing-campaign module 810 may include at least part of thereconnaissance module 816 carrying out the reconnaissance function code,the attack module 818 carrying out the attack function code, andoptionally the cleanup module 819 carrying out the cleanup functioncode.

System 800 further includes a sub-goals-selection module 820, includingone or more sub-goals-selection processors 822 and a sub-goals-selectionmemory 824, such as a non-transitory computer readable storage medium,having stored thereon instructions to be executed by the one or moresub-goals-selection processors 822. The instructions includeinstructions that, when carried out in the correct order, selectmultiple sub-goals to be protected in order to improve the security of atested networked system.

Turning additionally to FIGS. 12B and 12C, which, together, are aschematic block diagram of sub-goals-selection module 820, it is seenthat memory 824 includes multiple instructions to be executed by theprocessor(s) 822, including:

First instructions 825 to initialize a list L of sub-goals to beprotected to be an empty list;

Second instructions 826 to obtain a halting condition. The haltingcondition includes a Boolean condition to be applied to the list ofsub-goals;

Third instructions 827 to receive, from penetration-testing-campaignmodule 810, results of the one or more penetration tests of thenetworked system carried out by the penetration-testing-campaign module;

Fourth instructions 828 to determine, based on the received results,multiple paths of attack available to the attackers. As described indetail hereinabove, each such path of attack is an ordered sequence ofone or more attacker steps and one or more sub-goals;

Fifth instructions 829 to initialize a group Gr of relevant paths ofattack to consist of the determined multiple paths of attack;

Sixth instructions 830 to assign a calculated importance score, orimportance measure, to each of one or more sub-goals. Each of the one ormore sub-goals is included in at least one of the multiple paths ofattack included in the group Gr of relevant paths of attack.Additionally, for at least one given sub-goal of the one or moresub-goals, the importance score or measure assigned to the givensub-goal is based on the number of paths of attack in the group ofrelevant paths of attack which include the sub-goal;

Seventh instructions 832 to select one sub-goal included in at least onepath of attack in group Gr, and to add the selected sub-goal to the listL. The selection of the sub-goal is based on the importance scoreassigned to at least one of the one or more sub-goals;

Eighth instructions 833 to modify the group Gr of relevant paths ofattack by removing from group Gr every path of attack that includes theselected sub-goal added to list L by carrying out of instructions 832;

Ninth instructions 834 to evaluate the halting condition for the list Lof sub-goals;

Tenth instructions 835, to be carried out in response to determiningthat (i) the halting condition is not satisfied, and (ii) group Gr ofrelevant paths of attack includes at least one path of attack, to repeatthe sixth instructions 830 to the tenth instructions 835; and

Eleventh instructions 836, to be carried out in response to determiningthat (i) the halting condition is satisfied, or (ii) group Gr ofrelevant paths of attack is empty, to select one or more sub-goals fromthe list L of sub-goals.

Returning to FIG. 12A, it is seen that system 800 further includes areporting module 840, functionally associated with a user interface 850and with sub-goals-selection module 820. Reporting module 840 includesone or more reporting processors 842, and a reporting memory 844, suchas a non-transitory computer readable storage medium, having storedthereon instructions to be executed by the one or more reportingprocessors 842. The reporting memory 844 has stored:

instructions to receive from sub-goals-selection module 820 anidentification of one or more selected sub-goals from list L; and

instructions to provide a recommendation to protect the selected one ormore sub-goals, thereby to improve the security of the networked system.The instructions to provide the recommendation include at least one of:

-   -   (i) instructions to cause a display device (e.g. of a user        interface 850 or another display device, which may be located        remotely to the reporting module 840) to display information        about the recommendation;    -   (ii) instructions to store the information about the        recommendation in a file; and    -   (iii) instructions to electronically transmit the information        about the recommendation, for example using a transceiver 852        functionally associated with the reporting module 840.

In some embodiments, the penetration-testing-campaign memory 814,sub-goals-selection memory 824, and reporting memory 844 are each adedicated, and separate, memory component or storage medium. In otherembodiments, at least two of the penetration-testing-campaign memory814, the sub-goals-selection memory 824, and the reporting memory 844may be part of the same memory component or storage medium.

In some embodiments, the penetration-testing-campaign processor(s) 812,sub-goals-selection processor(s) 822, and reporting processor(s) 842 areeach dedicated, and separate, processors. In other embodiments, at leasttwo of the penetration-testing-campaign processor(s) 812,sub-goals-selection processor(s) 822, and reporting processor(s) 842share at least one common processor.

FIGS. 13A and 13B, together, are a flow chart of a method for providinga recommendation for improving the security of a network system againstattackers according to an embodiment of the second aspect of theinvention. The method of FIGS. 13A and 13B is described herein as beingcarried out using system 800 of FIG. 12A, but may be carried out usingany suitable system.

At step S900 shown in FIG. 13A, a penetration testing system, such assystem 800, carries out one or more tests of the networked system, forexample using testing methods known in the art.

At step S902, a list L of sub-goals to be protected is initialized to bean empty list, for example by execution of first instructions 825 ofsub-goals-selection module 820. In some embodiments, the list L is anordered list, such that the first element added to the list has thehighest priority, and subsequent elements have a monotonicallydecreasing priority.

A halting condition is obtained at step S904, for example by executionof second instructions 826 of sub-goals-selection module 820. Thehalting condition is or includes a Boolean condition to be applied tothe list L.

In some embodiments, the halting condition is true if and only if list Lconsists of one sub-goal. In some embodiments, the halting condition istrue if and only if list L consists of a pre-determined number ofsub-goals.

In some embodiments, the halting condition is true if and only if a sumof remediation costs of all members of list L satisfies a second Booleancondition. In some such embodiments, the second Boolean condition istrue if and only if the sum of remediation costs of all members of listL satisfies one of the conditions: higher than a pre-determinedthreshold, higher than or equal to the pre-determined threshold, lowerthan the pre-determined threshold, and lower than or equal to thepre-determined threshold.

It will be appreciated by people of skill in the art that step S900 mayoccur prior to steps S902 and S904, concurrently with steps S902 andS904, or after those steps.

Based on the results of the tests carried out at step S900, multiplepaths of attack which are available to attackers are determined at stepS906, for example by execution of third instructions 827 and fourthinstructions 828 of sub-goals-selection module 820. Each such path ofattack is an ordered sequence of one or more attacker steps and one ormore sub-goals.

In some embodiments, each path of attack determined in step S906 startsat a starting sub-goal that is assumed to be achievable by the attacker,and ends at a final sub-goal which is assumed to be a goal of theattacker in at least one of the tests carried out at step S900. Forexample, such paths of attack are illustrated in FIGS. 3A-9Bhereinabove.

In some embodiments, in each path of attack determined in step S906: (A)each attacker step in the path of attack is preceded by a sub-goal andfollowed by a sub-goal in the path of attack, (B) each sub-goal in thepath of attack, except for a starting sub-goal of the path of attack, ispreceded by an attacker step in the given path of attack, and (C) eachsub-goal in the path of attack, except for a final sub-goal of the pathof attack, is followed by an attacker step in the path of attack.

In some embodiments, at an optional step S908 a, the paths of attackdetermined at step S906 are represented by a graph, for example byattacker-steps-selection module 820. Each determined path of attackcorresponds to a path in the graph.

In some such embodiments, for each given path in the graph correspondingto a specific path of attack available to the attackers, all sub-goalsand all attacker steps included in the specific path of attack arerepresented as graph nodes included in the given path in said graph.Such a representation is illustrated, for example, in FIGS. 3A, 4, 5, 6,9A and 9B.

In some other such embodiments, for each given path in the graphcorresponding to a specific path of attack available to the attackers,all sub-goals included in the specific path of attack are represented asgraph nodes included in the given path in the graph and all attackersteps included in the specific path of attack are represented as graphedges included in the given path in the graph. Such a representation isillustrated, for example, in FIGS. 3B, and 7A to 8.

In some embodiments, at an optional step S908 b, the paths of attackdetermined at step S906 are represented by a list, for example byattacker-steps-selection module 820. Each given path of attackcorresponds to an item in the list that includes all sub-goals and allattacker steps included in the given path of attack. Such arepresentation is illustrated, for example, in FIG. 3C.

At step S910, a group Gr of relevant paths of attack is initialized toconsist of all the multiple paths of attack determined at step S906, forexample by execution of fifth instructions 829 of sub-goals-selectionmodule 820.

A calculated importance score, or importance measure, is assigned toeach of one or more sub-goals at step S912, for example by execution ofsixth instructions 830 of sub-goals-selection module 820. Each sub-goalof the one or more sub-goals to which an importance score is assigned,is included in at least one of the paths of attack included in group Grof relevant paths of attack. The importance score assigned to at leastone given sub-goal of the one or more sub-goals is based on a number ofpaths of attack in group Gr of relevant paths of attack which includethat given sub-goal.

In some embodiments, the importance score assigned, at step S912, toeach given sub-goal of the one or more sub-goals, is based on the numberof paths of attack in group Gr of relevant paths of attack which includethat given sub-goal.

In some embodiments, the importance score assigned, at step S912, to atleast one of the one or more sub-goals, is based only on the number ofpaths of attack in group Gr of relevant paths of attack which includethat sub-goal.

In some embodiments, the importance score assigned, at step S912, toeach given sub-goal of the one or more sub-goals, is based only on thenumber of paths of attack in group Gr of relevant paths of attack whichinclude the given sub-goal.

In some embodiments, the importance score assigned, at step S912, to atleast one of the one or more sub-goals, is equal to the number of pathsof attack in group Gr of relevant paths of attack which include thatsub-goal.

In some embodiments, the importance score assigned, at step S912, to atleast one given sub-goal of the one or more sub-goals, is based on (i)the number of paths of attack in group Gr of relevant paths of attackwhich include the given sub-goal, and (ii) at least one member selectedfrom the group consisting of: a cost of remediation of the givensub-goal, a cost of exploitation of an attacker step leading to thegiven sub-goal, a cost of remediation of an attacker step leading to thegiven sub-goal, and a probability of success of an attacker step leadingto the given sub-goal.

In some embodiments, the importance scores assigned at step S912 aresuch that a higher numerical value of the importance score for aspecific sub-goal is indicative of a higher degree of importanceassociated with the specific sub-goal.

In some other embodiments, the importance scores assigned at step S912are such that a lower numerical value of the importance score for aspecific sub-goal is indicative of a higher degree of importanceassociated with the specific sub-goal.

At step S916, one sub-goal included in at least one of the paths ofattack in group Gr is selected and is added to the list L of sub-goals,for example by execution of seventh instructions 832 ofsub-goals-selection module 820. The selection of the sub-goal is basedon the importance score assigned to at least one of the one or moresub-goals (which are included in at least one of the paths of attack ingroup Gr).

In some embodiments, the selected sub-goal is one whose assignedimportance score meets a predefined criterion.

In some embodiments, the selected sub-goal is one whose assignedimportance score has the highest numerical value of all importancescores assigned to sub-goals included in at least one of the paths ofattack in group Gr of relevant paths of attack. In other embodiments,the selected sub-goal is one whose assigned importance score has thelowest numerical value of all importance scores assigned to sub-goalsincluded in at least one of the paths of attack in group Gr of relevantpaths of attack.

In some embodiments, it may happen that multiple sub-goals have assignedthereto a common importance score, which is an importance scoreindicative of the highest degree of importance of all the importancescores assigned to the multiple sub-goals at step S912. In suchembodiments, a tie-breaking rule is used to select one of those multiplesub-goals as the one sub-goal.

In some such embodiments, the tie-breaking rule depends on at least oneof:

-   -   a cost of remediation of at least one of the multiple sub goals        having assigned the common importance score;    -   a cost of exploitation of at least one attacker step leading to        at least one of the multiple sub-goals having assigned the        common importance score;    -   a cost of remediation of a at least one attacker step leading to        at least one of the multiple sub-goals having assigned the        common importance score; and    -   a probability of success of at least one attacker step leading        to at least one of the multiple sub-goals having assigned the        common importance score.

In some other such embodiments, the tie-breaking rule includes randomlyselecting one of the multiple sub-goals having assigned the commonimportance score as the one sub-goal.

In some embodiments, in which list L is an ordered list, the selectedsub-goal is added to the end of list L.

At step S918 the group Gr of relevant paths of attack is modified, forexample by execution of eighth instructions 833 of sub-goals-selectionmodule 820. The modification of group Gr includes removing from thegroup Gr every path of attack that includes the selected sub-goal (ofstep S916). Thus, following step S918, group Gr typically includes fewerelements than prior to step S918.

At step S920, the halting condition for list L (obtained at step S904)is evaluated, for example by execution of ninth instructions 834 ofsub-goals-selection module 820.

If the halting condition is not satisfied, at step S922 the group Gr isevaluated to determine whether it includes at least one path of attack.

If the group Gr includes at least one path of attack, another iterationof the method is initiated, by returning the flow to step S912, forexample by execution of tenth instructions 835 of sub-goals-selectionmodule 820.

If at step S920 it is determined that the halting condition issatisfied, or if at step S922 it is determined that the group Gr is anempty group (i.e. includes zero paths of attack), one or more of thesub-goals in list L are selected as sub-goals to be protected at stepS924, for example by execution of eleventh instructions 836 ofsub-goals-selection module 820.

At step S926, a recommendation is provided to protect one or more of thesub-goals currently in list L, thereby to improve the security of thenetworked system. For example, the recommendation may be provided byreporting module 840. The recommendation may be provided by any one ormore of: (i) causing a display device to display information about therecommendation, (ii) recording the information about the recommendationin a file, and (iii) electronically transmitting the information aboutthe recommendation, for example to a remote location.

In some embodiments, in which list L is an ordered list, and at stepS916 sub-goals are added to the end of list L such that newly addedsub-goals become the last elements in the ordered list L, therecommendation provided at step S926 includes a recommendation toprotect the one or more sub-goals selected from list L according to anorder of the sub-goals in the list L.

DEFINITIONS

This disclosure should be interpreted according to the definitionsbelow.

In case of a contradiction between the definitions in this Definitionssection and other sections of this disclosure, this section shouldprevail.

In case of a contradiction between the definitions in this section and adefinition or a description in any other document, including in anotherdocument included in this disclosure by reference, this section shouldprevail, even if the definition or the description in the other documentis commonly accepted by a person of ordinary skill in the art.

-   1. “computing device”—Any device having a processing unit into which    it is possible to install code that can be executed by the    processing unit. The installation of the code may be possible even    while the device is operative in the field or it may be possible    only in the factory.-   2. “peripheral device”—Any device, whether a computing device or    not, that provides input or output services to at least one other    device that is a computing device. Examples of peripheral devices    are printers, plotters, scanners, environmental sensors, smart-home    controllers, digital cameras, speakers and display screens. A    peripheral device may be directly connected to a single computing    device or may be connected to a communication system through which    it can communicate with one or more computing devices. A storage    device that is (i) not included in or directly connected to a single    computing device, and (ii) accessible by multiple computing devices,    is a peripheral device.-   3. “network” or “computing network”—A collection of computing    devices and peripheral devices which are all connected to common    communication means that allow direct communication between any two    of the devices without requiring passing the communicated data    through a third device. The network includes both the connected    devices and the communication means. A network may be wired or    wireless or partially wired and partially wireless.-   4. “networked system” or “networked computing system”—One or more    networks that are interconnected so that communication is possible    between any two devices of the one or more networks, even if they do    not belong to the same network. The connection between different    networks of the networked system may be achieved through dedicated    computing devices, and/or through computing devices that belong to    multiple networks of the networked system and also have other    functionality in addition to connecting between networks. The    networked system includes the one or more networks, any connecting    computing devices and also peripheral devices accessible by any    computing device of the networked system. Note that a single network    is a networked system having only one network, and therefore a    network is a special case of a networked system.-   5. “module”—A portion of a system that implements a specific task. A    module may be composed of hardware, software or any combination of    both. For example, in a module composed of both hardware and    software, the hardware may include a portion of a computing device,    a single computing device or multiple computing devices, and the    software may include software code executed by the portion of the    computing device, by the single computing device or by the multiple    computing devices. A computing device associated with a module may    include one or more processors and computer readable storage medium    (non-transitory, transitory or a combination of both) for storing    instructions or for executing instructions by the one or more    processors.-   6. “network node of a networked system” or “node of a networked    system”—Any computing device or peripheral device that belongs to    the networked system.-   7. “security vulnerability of a network node” or “vulnerability of a    network node”—A weakness which allows an attacker to compromise the    network node. A vulnerability of a network node may be caused by one    or more of a flawed configuration of a component of the network    node, a flawed setting of a software module in the network node, a    bug in a software module in the network node, a human error while    operating the network node, having trust in an already-compromised    other network node, and the like.    -   A weakness that allows an attacker to compromise a network node        only conditionally, depending on current conditions in the        network node or in the networked system in which the network        node resides, is still a vulnerability of the network node, but        may also be referred to as a “potential vulnerability of the        network node”. For example, a vulnerability that compromises any        network node running the Windows 7 Operating System, but only if        the network node receives messages through a certain Internet        port, can be said to be a vulnerability of any Windows 7 network        node, and can also be said to be a potential vulnerability of        any such node. Note that in this example the potential        vulnerability may fail in compromising the node either because        the certain port is not open (a condition in the node) or        because a firewall is blocking messages from reaching the        certain port in the node (a condition of the networked system).-   8. “security vulnerability of a networked system” or “vulnerability    of a networked system”—A weakness which allows an attacker to    compromise the networked system. A vulnerability of a networked    system may be caused by one or more of a vulnerability of a network    node of the networked system, a flawed configuration of a component    of the networked system, a flawed setting of a software module in    the networked system, a bug in a software module in the networked    system, a human error while operating the networked system, and the    like.    -   A weakness that allows an attacker to compromise a networked        system only conditionally, depending on current conditions in        the networked system, is still a vulnerability of the networked        system, but may also be referred to as a “potential        vulnerability of the networked system”. For example, if a        network node of the networked system has a potential        vulnerability then that vulnerability can be said to be a        vulnerability of the networked system, and can also be said to        be a potential vulnerability of the networked system.-   9. “validating a vulnerability” or “validating a potential    vulnerability” (for a given network node or for a given networked    system)—Verifying that the vulnerability compromises the given    network node or the given networked system under the conditions    currently existing in the given network node or the given networked    system.    -   The validation of the vulnerability may be achieved by actively        attempting to compromise the given network node or the given        networked system and then checking if the compromising attempt        was successful. Such validation is referred to as “active        validation”. Alternatively, the validation of the vulnerability        may be achieved by simulating the exploitation of the        vulnerability or by otherwise evaluating the results of such        exploitation without actively attempting to compromise the given        network node or the given networked system. Such validation is        referred to as “passive validation”. Note that just assuming        that a vulnerability will succeed in compromising a given        network node or a given networked system under current        conditions without executing either active validation or passive        validation, is not considered as validating the vulnerability.-   10. “vulnerability management”—A cyclical practice of identifying,    classifying, remediating, and mitigating vulnerabilities of network    nodes in a networked system.-   11. “penetration testing” or “pen testing” (in some references also    known as “red team assessment” or “red team testing”, but in other    references those terms referring to a red team have a different    meaning than “penetration testing”)—A process in which a networked    system is evaluated in order to determine if it can be compromised    by an attacker by utilizing one or more security vulnerabilities of    the networked system. If it is determined that the networked system    can be compromised, then the one or more security vulnerabilities of    the networked system are identified and reported.    -   Unlike a vulnerability management process which operates at the        level of isolated vulnerabilities of individual network nodes, a        penetration test may operate at a higher level which considers        vulnerabilities of multiple network nodes that might be jointly        used by an attacker to compromise the networked system.    -   A penetration testing process involves at least the following        functions: (i) a reconnaissance function, (ii) an attack        function, and (iii) a reporting function. It should be noted        that the above functions do not necessarily operate sequentially        according to the above order, but may operate in parallel or in        an interleaved mode.    -   Unless otherwise explicitly specified, a reference to        penetration testing should be understood as referring to        automated penetration testing.-   12. “automated penetration testing”—Penetration testing in which at    least one of the reconnaissance function, the attack function and    the reporting function is at least partially automated.-   13. “penetration testing system”—A system capable of performing    penetration testing, regardless if composed of hardware, software or    combination of both.-   14. “reconnaissance function” or “recon function”—The function in a    penetration testing process that handles collection of data about    the tested networked system. The collected data may include internal    data of one or more network nodes of the tested networked system.    Additionally, the collected data may include data about    communication means of the tested networked system and about    peripheral devices of the tested networked system.    -   The collected data may also include data that is only indirectly        related to the tested networked system, for example business        intelligence data about the organization owning the tested        networked system, collected in order to use it for assessing        importance of resources of the networked system.    -   The functionality of a reconnaissance function may be        implemented by any combination of (i) software executing in a        remote computing device, where the remote computing device may        probe the tested networked system for the purpose of collecting        data about it, (ii) hardware and/or software simulating or        duplicating the tested networked system, (iii) a reconnaissance        agent software module executing in one or more network nodes of        the tested networked system.-   15. “attack function”—The function in a penetration testing process    that handles determination of whether one or more security    vulnerabilities exist in the tested networked system. The    determination is based on data collected by the reconnaissance    function of the penetration testing. The attack function generates    data about each of the identified security vulnerabilities, if any.    -   The functionality of an attack function may be implemented by        any combination of (i) software executing in a remote computing        device, where the remote computing device may attack the tested        networked system for the purpose of verifying that it can be        compromised, (ii) hardware and/or software simulating or        duplicating the tested networked system, (iii) an attack agent        software module executing in one or more network nodes of the        tested networked system.    -   The methods used by an attack function may include executing a        real attack on the tested networked system by attempting to        change at least one setting, mode or state of a network node or        of a hardware or software component of a network node, in order        to verify that the tested networked system may be compromised.        In such case, the attempt may result in actually compromising        the tested networked system. Alternatively, the methods used by        an attack function may be such that whenever there is a need to        verify whether a setting, a mode or a state of a network node or        of a hardware or software component of a network node can be        changed in a way that compromises the tested networked system,        the verification is done by simulating the effects of the change        or by otherwise evaluating them without ever actually        compromising the tested networked system.-   16. “reporting function”—The function in a penetration testing    process that handles reporting of results of the penetration    testing. The reporting comprises at least one of (i) causing a    display device to display a report including information about the    results of the penetration testing, (ii) recording a report    including information about the results of the penetration testing    in a file, and (iii) electronically transmitting a report including    information about the results of the penetration testing.    -   The functionality of a reporting function may be implemented by        software executing in a remote computing device, for example in        the computing device implementing the attack function of the        penetration testing.-   17. “recovery function” or “clean-up function”—The function in a    penetration testing process that handles cleaning-up after a    penetration test. The recovery includes undoing any operation done    during the penetration testing process that results in compromising    the tested networked system.    -   The functionality of a recovery function may be implemented by        any combination of (i) software executing in a remote computing        device, for example in the computing device implementing the        attack function of the penetration testing, (ii) an attack agent        software module executing in one or more network nodes of the        tested networked system.-   18. “a campaign of penetration testing” or “penetration testing    campaign” or just “campaign”—A specific run of a specific test of a    specific networked system by the penetration testing system.    -   An execution of a campaign must end by one of the following: (i)        determining by the penetration testing system that the goal of        the attacker was reached by the campaign, (ii) determining by        the penetration testing system that the goal of the attacker        cannot be reached by the campaign, (iii) if the campaign is        assigned a time limit, exceeding the time limit by the campaign,        and (iv) manually terminating the campaign by a user of the        penetration testing system.-   19. “results of a penetration testing campaign”—Any output generated    by the penetration testing campaign. This includes, among other    things, data about any security vulnerability of the networked    system tested by the penetration testing campaign that is detected    by the campaign. It should be noted that in this context the word    “results” is used in its plural form regardless of the amount of    output data generated by the penetration testing campaign, including    when the output consists of data about a single security    vulnerability.-   20. “information item of a campaign”—A variable data item that a    penetration testing system must know its value before executing the    campaign. Note that a data item must be able to have different    values at different campaigns in order to be considered an    information item of the campaign. If a data item always has the same    value for all campaigns, it is not an information item of the    campaign, even if it must be known and is being used by the    penetration testing system when executing the campaign.    -   A type of an attacker and a goal of an attacker are examples of        information items of a campaign. Another example of an        information item of a campaign that is more complex than the        previous two simple examples is a subset of the network nodes of        the networked system that is assumed to be already compromised        at the time of beginning the penetration testing campaign, with        the subset defined either by an explicit selection of network        nodes or by a Boolean condition each node of the subset has to        satisfy.    -   A value of an information item may be composed either of a        simple value or of both a main value and one or more auxiliary        values. If a specific main value of an information item requires        one or more auxiliary values that complete the full        characterization of the value, then the combination of the main        value and the one or more auxiliary values together is        considered to be the value assigned to the information item. For        example, for a “goal of the attacker” information item, after a        user selects a main value of “exporting a specific file from        whatever node having a copy of it”, the user still has to        provide a file name as an auxiliary value in order for the goal        information item to be fully characterized. In this case the        combination of “exporting a specific file from whatever node        having a copy of it” and the specific file name is considered to        be the value of the “goal of the attacker” information item.-   21. “specifications of a campaign” or “scenario”—A collection of    values assigned to all information items of the campaign. As having    a value for each information item of a campaign is essential for    running it, a campaign of a penetration testing system cannot be run    without providing the penetration testing system with full    specifications of the campaign. A value of an information item    included in the specifications of a campaign may be manually    selected by a user or may be automatically determined by the    penetration testing system. In the latter case, the automatic    determination by the system may depend on one or more values    selected by the user for one or more information items of the    campaign, or it may be independent of any selection by the user. For    example, the selection of the capabilities of the attacker may    automatically be determined by the system based on the user-selected    type of the attacker, and the lateral movement strategy of the    attacker may be automatically determined by the system independently    of any user selection.-   22. “attacker” or “threat actor”—An entity, whether a single person,    a group of persons or an organization, that might conduct an attack    against a networked system by penetrating it for uncovering its    security vulnerabilities and/or for compromising it.-   23. “a type of an attacker”—A classification of the attacker that    indicates its main incentive in conducting attacks of networked    systems. Typical values for a type of an attacker are    state-sponsored, opportunistic cyber criminal, organized cyber    criminal and insider.    -   An attacker can have only a single type.-   24. “a capability of an attacker”—A tool in the toolbox of the    attacker. A capability describes a specific action that the attacker    can perform. Examples of capabilities are copying a local file of a    network node and exporting it to the attacker out of the networked    system and remotely collecting database information from an SQL    server of the networked system. In some systems, selecting a type of    an attacker causes a corresponding default selection of capabilities    for that type of attacker, but the user may have an option to    override the default selection and add or delete capabilities.    -   An attacker can have one or multiple capabilities.-   25. “a goal of an attacker”—What the attacker of a campaign is    trying to achieve when attacking a targeted networked system. In    other words, what is the criterion according to which the attacker    will judge whether the attack was a success or a failure and/or to    what extent was it a success or a failure. Selecting a type of an    attacker may cause a default selection of a goal for that attacker,    but the user may have an option to override the default selection.    An attacker can have one or multiple goals.-   26. “penetration testing by simulation” or “simulated penetration    testing”—Penetration testing in which the methods used by the attack    function are such that whenever there is a need to verify whether a    setting, a mode or a state of a network node or of a hardware or    software component of a network node can be changed in a way that    compromises the tested networked system, the verification is done by    simulating the effects of the change or by otherwise evaluating them    without risking compromising the tested networked system.-   27. “penetration testing by actual attack” or “actual attack    penetration testing” or “penetration testing by actual exploit” or    “actual exploit penetration testing”—Penetration testing in which    the methods used by the attack function include executing a real    attack on the tested networked system by attempting to change at    least one setting, mode or state of a network node or of a hardware    or software component of a network node in order to verify that the    tested networked system may be compromised, such that the attempt    may result in compromising the tested networked system.-   28. “penetration testing by reconnaissance agents” or    “reconnaissance agent penetration testing”—Penetration testing in    which the functionality of the reconnaissance function is at least    partially implemented by a reconnaissance agent software module    installed and executed in each one of multiple network nodes of the    tested networked system.-   29. “reconnaissance client agent”, “reconnaissance agent” or “recon    agent”—A software module that can be installed on a network node and    can be executed by a processor of that network node for partially or    fully implementing the reconnaissance function of a penetration    test. A reconnaissance agent must be capable, when executed by a    processor of the network node in which it is installed, of    collecting data at least about some of the events occurring in the    network node. Such events may be internal events of the network node    or messages sent out of the network node or received by the network    node. A reconnaissance agent may be capable of collecting data about    all types of internal events of its hosting network node.    Additionally, it may be capable of collecting other types of data of    its hosting network node. A reconnaissance agent may additionally be    capable of collecting data about other network nodes or about other    components of a networked system containing the hosting network    node. A reconnaissance agent may be persistently installed on a    network node, where “persistently” means that once installed on a    network node the reconnaissance agent survives a reboot of the    network node. Alternatively, a reconnaissance agent may be    non-persistently installed on a network node, where    “non-persistently” means that the reconnaissance agent does not    survive a reboot of the network node and consequently should be    installed again on the network node for a new penetration test in    which the network node takes part, if the network node was rebooted    since the previous penetration test in which it took part.-   30. “attack client agent” or “attack agent”—A software module that    can be installed on a network node and can be executed by a    processor of that network node for partially or fully implementing    the attack function of a penetration test. Typically, an attack    agent is installed by an actual attack penetration testing system in    a network node that it had succeeded to compromise during a    penetration test. Once installed on such network node, the attack    agent may be used as a tool for compromising other network nodes in    the same networked system. In such case, the attack agent may    include code that when executed by a processor of the compromised    network node compromises another network node that is adjacent to it    in the networked system, possibly taking advantage of the high level    of trust it may have from the point of view of the adjacent network    node. Another type of an attack agent may include code that when    executed by a processor of a network node determines whether that    network node would be compromised if a given operation is performed.-   31. “penetration testing software module” or “remote computing    device penetration testing software module”—A software module that    implements the full functionality of a penetration testing system,    except for the functionality implemented by (i) reconnaissance    agents, (ii) attack agents, and (iii) hardware and/or software    simulating or duplicating the tested networked system, if such    components are used in the implementation of the penetration testing    system.    -   The penetration testing software module may be installed and        executed on a single computing device or comprise multiple        software components that reside on multiple computing devices.        For example, a first component of the penetration testing        software module may implement part or all of the reconnaissance        function and be installed and executed on a first computing        device, a second component of the penetration testing software        module may implement part or all of the attack function and be        installed and executed on a second computing device, and a third        component of the penetration testing software module may        implement the reporting function and be installed and executed        on a third computing device.-   32. “internal data of a network node”—Data related to the network    node that is only directly accessible to code executing by a    processor of the network node and is only accessible to any code    executing outside of the network node by receiving it from code    executing by a processor of the network node. Examples of internal    data of a network node are data about internal events of the network    node, data about internal conditions of the network node, and    internal factual data of the network node.-   33. “internal event of/in a network node”—An event occurring in the    network node whose occurrence is only directly detectable by code    executing by a processor of the network node. Examples of an    internal event of a network node are an insertion of a USB drive    into a port of the network node, and a removal of a USB drive from a    port of the network node. An internal event may be a free event or a    non-free event.    -   It should be noted that the term “an event of X” refers to any        occurrence of an event of the type X and not to a specific        occurrence of it. For referring to a specific occurrence of an        event of type X one should explicitly say “an occurrence of        event of X”. Thus, a software module which looks for detecting        insertions of a USB drive into a port is “detecting an event of        USB drive insertion”, while after that module had detected such        event it may report “an occurrence of an event of USB drive        insertion”.-   34. “internal condition of/in a network node”—A Boolean condition    related to the network node which can only be directly tested by    code executing by a processor of the network node. Examples of an    internal condition of a network node are whether the local disk of    the terminal node is more than 98% full or not, and whether a USB    drive is currently inserted in a port of the network node.-   35. “internal factual data of/in a network node” or “internal facts    of a network node”—Facts related to the network node which can only    be directly found by code executing by a processor of the network    node. Examples of factual data of a network node are the version of    the firmware of a solid-state drive installed in the network node,    the hardware version of a processor of the network node, and the    amount of free space in a local disk of the network node.-   36. “resource of a network node”—A file in the network node, a    folder in the network node, credentials of a user residing in the    network node (the credentials not necessarily applying to the    network node containing the credentials), a peripheral device of the    network node or a communication device accessible to the network    node.-   37. “resource of a networked system”—A file in a network node of the    networked system, a folder in a network node of the networked    system, credentials of a user of the networked system, a peripheral    device of a network node of the networked system, a peripheral    device directly attached to a network of the networked system, or a    communication device accessible by a network node of the networked    system.-   38. “access rights” (of a user in a network node)—Rights of the user    to perform operations on resources of the network node. For example,    a right to execute a given file or a given class of files, a right    to read from a given file or from a given folder, a right to create    a new file in a given folder, a right to change a given file, a    right to print on a given printer, or a right to send out data    through a given communication device.    -   Access rights may be conditioned on the user authenticating        himself before getting the rights to perform the relevant        operations. A user is said to have certain access rights        regardless if those rights are conditioned on authentication or        not.    -   The term “access rights” in the plural may be used even if only        a single right is involved (e.g. when a user has only a right to        read a single file in the network node).-   39. “user credentials”—An attestation issued to the user for    authenticating himself in order to be allowed to use access rights    granted to him in one or more network nodes. User credentials may    include a user name, a user ID, a password, any combination of the    three, or any other data item which is expected not to be available    to other people.-   40. “compromising a network node”—Successfully causing execution of    an operation in the network node that is not allowed for the entity    requesting the operation by the rules defined by an administrator of    the network node, or successfully causing execution of code in a    software module of the network node that was not predicted by the    vendor of the software module. Examples for compromising a network    node are reading a file without having read permission for it,    modifying a file without having write permission for it, deleting a    file without having delete permission for it, exporting a file out    of the network node without having permission to do so, getting an    access right higher than the one originally assigned without having    permission to get it, getting a priority higher than the one    originally assigned without having permission to get it, changing a    configuration of a firewall network node such that it allows access    to other network nodes that were previously hidden behind the    firewall without having permission to do it, and causing execution    of software code by utilizing a buffer overflow. As shown by the    firewall example, the effects of compromising a certain network node    are not necessarily limited to that certain network node. In    addition, executing successful ARP spoofing, denial-of-service,    man-in-the-middle or session-hijacking attacks against a network    node are also considered compromising that network node, even if not    satisfying any of the conditions listed above in this definition.-   41. “ARP spoofing”—a technique for compromising a target network    node in which an attacker sends a false Address Resolution Protocol    (ARP) reply message to the target network node. The aim is to    associate an attacker's MAC address (either a MAC address of the    node sending the false ARP reply message or a MAC address of another    node controlled by the attacker) with the IP address of another    host, such as the default gateway, causing any traffic sent by the    target node and meant for that IP address to be sent to the attacker    instead. ARP spoofing may allow an attacker to intercept data frames    on a network, modify the traffic, or stop all traffic to a certain    node. Often the attack is used as an opening for other attacks, such    as denial-of-service, man-in-the-middle, or session-hijacking    attacks.-   42. “denial-of-service attack”—a cyber-attack where an attacker    seeks to make a service provided by a network node to other network    nodes unavailable to its intended users either temporarily or    indefinitely. The denial-of-service attack may be accomplished by    flooding the node providing the targeted service with superfluous    requests in an attempt to overload it and prevent some or all    legitimate requests from being fulfilled. Alternatively, the    denial-of-service attack may be accomplished by causing some or all    of the legitimate requests addressed to the targeted service to not    reach their destination.-   43 “man-in-the-middle attack”—a cyber-attack where an attacker    secretly relays and possibly alters the communication between two    network nodes who believe they are directly communicating with each    other. One example of man-in-the-middle attacks is active    eavesdropping, in which the attacker makes independent connections    with the victims and relays messages between them to make them    believe they are communicating directly with each other, when in    fact the entire communication session is controlled by the attacker.    The attacker must be able to intercept all relevant messages passing    between the two victims and inject new ones.-   44. “session-hijacking attack”—a cyber-attack where a valid    communication session between two network nodes in a networked    system is used by an attacker to gain unauthorized access to    information or services in the networked computer system.-   45. “compromising a networked system”—Compromising at least one    network node of the networked system or successfully causing    execution of an operation in the networked system that is not    allowed for the entity requesting the operation by the rules defined    by an administrator of the networked system. Examples for operations    in the networked system that may not be allowed are exporting a file    out of the networked system without having permission to do so,    sending a file to a network printer without having permission to do    so, and copying a file from one network node to another network node    without having permission to do so.-   46. “compromising a software application”—Successfully causing the    software application to execute an operation that is not allowed for    the entity requesting the operation by the rules defined by an    administrator of the network node on which the software application    is installed or by a vendor of the software application, or    successfully causing the execution of code in the software    application that was not predicted by the vendor of the software    application. Examples for compromising a software application are    changing a configuration file controlling the operation of the    software application without having permission for doing so, and    activating a privileged function of the software application without    having permission for doing so. In addition, causing the software    application to execute a macro without checking rights of the macro    code to do what it is attempting to do is also considered    compromising that software application, even if not satisfying any    of the conditions listed above in this definition.-   47 “administrator of a network node”—Any person that is authorized,    among other things, to define or change at least one rule    controlling at least one of an access right, a permission, a    priority and a configuration in the network node.-   48 “administrator of a networked system”—Any person that is    authorized, among other things, to define or change at least one    rule controlling at least one of an access right, a permission, a    priority and a configuration in the networked system. Note that an    administrator of a networked system may also be an administrator of    one or more of the network nodes of the networked system.-   49. “achieving full control of a computing device”—For a multi-user    computing device that distinguishes between administrator and    non-administrator users, logging into the computing device with    administrator access rights. For a single-user computing device,    logging into the computing device as the single user.-   50. “remote computing device” or “penetration testing remote    computing device” (with respect to a given networked system)—A    computing device that executes software implementing part or all of    the penetration testing software module that is used for testing the    given networked system.    -   A remote computing device may be (i) outside of the given        networked system, or (ii) inside the given networked system. In        other words, a remote computing device is not necessarily        physically remote from the given networked system. It is called        “remote” to indicate its functionality is logically separate        from the functionality of the given networked system.    -   A remote computing device may (i) be a dedicated computing        device that is dedicated only to doing penetration testing,        or (ii) also implement other functionality not directly related        to penetration testing.    -   A remote computing device is not limited to be a single physical        device with a single processing unit. It may be implemented by        multiple separate physical devices packaged in separate packages        that may be located at different locations. Each of the separate        physical devices may include one or multiple processing units.    -   A remote computing device may be (i) a physical computing        device, or (ii) a virtual machine running inside a physical        computing device on top of a hosting operating system.-   51. “explicitly selecting”—Directly and clearly selecting, by a    human user, of one option out of multiple options available to the    human user, leaving no room for doubt and not relying on making    deductions by a computing device.    -   Examples of explicit selections are (i) selection of a specific        type of an attacker from a drop-down list of types, (ii)        selection of specific one or more attacker capabilities by        marking one or more check boxes in a group of multiple check        boxes corresponding to multiple attacker capabilities, and (iii)        reception for viewing by a user of a recommendation        automatically computed by a computing device for a value of an        information item and actively approving by the user of the        recommendation for using the value, provided that the approving        user has an option of rejecting the recommendation and selecting        a different value for the information item.    -   Examples of selections that are not explicit selections are (i)        selection of specific one or more attacker capabilities by        selecting a specific scenario of a penetration testing system        from a pre-defined library of scenarios, where the specific        scenario includes an attacker having the one or more        capabilities, and (ii) selection of specific one or more        attacker capabilities by selecting a specific goal of an        attacker, accompanied by a deduction by a computing device        concluding that the specific one or more attacker capabilities        must be selected because they are essential for the attacker to        succeed in meeting the specific goal.-   52. “automatically selecting”—Selecting, by a computing device, of    one option out of multiple options, without receiving from a human    user an explicit selection of the selected option. It should be    noted that the selecting of an option is an automatic selecting even    if the computing device is basing the selection on one or more    explicit selections by the user, as long as the selected option    itself is not explicitly selected by the user. It should also be    noted that receiving from a user of an approval for a recommendation    which is otherwise automatically selected without giving the user an    ability to override the recommendation does not make the selection a    non-automatic selection.    -   An example of an automatic selection is a selection by a        computing device of one or more attacker capabilities by (a)        receiving from a user an explicit selection of a specific        scenario of a penetration testing system from a pre-defined        library of scenarios, (b) determining by the computing device        that the specific scenario includes an attacker having the one        or more capabilities, and (c) deducing by the computing device        that the user wants to select the one or more attacker        capabilities.    -   An example of a selection that is not an automatic selection is        a selection of a value for an information item by (a)        calculating by a computing device of a recommended value for the        information item, (b) displaying the recommendation to a user,        and (c) receiving from the user an explicit approval to use the        recommended value of the information item, provided that the        approving user has an option of rejecting the recommendation and        selecting a different value for the information item.-   53. “defensive application”—A software application whose task is to    defend the network node in which it is installed against potential    attackers. A defensive application may be a passive defensive    application, in which case it only detects and reports penetration    attempts into its hosting network node but does not attempt to    defend against the detected attacks. Alternatively, a defensive    application may be an active defensive application, in which case it    not only detects penetration attempts into its hosting network node    but also attempts to defend its hosting node against the detected    attacks by activating at least one counter-measure.-   54. “user interface”—A man-machine interface that does at least one    of (i) providing information to a user, and (ii) receiving input    from the user. Towards this end, any user interface includes at    least one of (i) an input device (e.g. touch-screen, mouse,    keyboard, joystick, camera) for receiving input from the user,    and (ii) an output device (e.g. display screen such as a    touch-screen, speaker) for providing information to the user. A user    interface typically also includes executable user-interface code for    at least one of (i) causing the output device to provide information    to the user (e.g. to display text associated with radio-buttons or    with a check list, or text of a drop-down list) and (ii) processing    user-input received via the input device.    -   In different examples, the executable code may be compiled-code        (e.g. in assembly or machine-language), interpreted byte-code        (e.g. Java byte-code), or browser-executed code (e.g. JavaScript        code) that may be sent to a client device from a remote server        and then executed by the client device.-   55. “user interface of a computing device”—A user interface that is    functionally attached to the computing device and serves the    computing device for interacting with the user.    -   An input device of a user interface of a computing device may        share a common housing with the computing device (e.g. a        touch-screen of a tablet), or may be physically separate from        the computing device and be in communication with it, either        through a physical port (e.g. a USB port) or wirelessly (e.g. a        wireless mouse).    -   An output device of a user interface of a computing device may        share a common housing with the computing device (e.g. a        touch-screen of a tablet), or may be physically separate from        the computing device and be in communication with it, either        through a physical port (e.g. an HDMI port) or wirelessly.    -   User-interface code of a user interface of a computing device is        stored in a memory accessible to the computing device and is        executed by one or more processors of the computing device. In        one example related to web-based user interfaces, at least some        of this code may be received from a remote server and then        locally executed by the computing device which functions as a        client. In another example related to locally-implemented user        interfaces, all of the user-interface code is pre-loaded onto        the computing device.-   56. “random selection”—A selection that depends on a random or    pseudo-random factor.

Different possible outcomes in a random selection do not necessarilyhave the same probabilities of being selected.

-   57. “broadcast domain”—A logical division of a networked system, in    which all network nodes can reach each other by broadcasting at the    data link layer. In other words, each network node in a broadcast    domain can transmit a data link broadcast message that is addressed    to all other network nodes within its broadcast domain.-   58. “attacker step”—One or more actions performed by an attacker of    a networked system in order to achieve a certain result. An attacker    step may be included in an actual or potential attempt of an    attacker to compromise a networked system that includes one or more    attacker steps. Performing a given attacker step may be conditioned    on certain achievements being already achieved by the attacker prior    to carrying out the given attacker step.    -   An example of an attacker step that consists of a single action        is the recovering of a password enabling access to a given        network node from a known hash code (e.g. using a pre-compiled        dictionary of hash codes and their corresponding passwords, when        the algorithm of hashing is known). In this example, the        attacker step is conditioned on a prior achievement by the        attacker of finding out the hash code.    -   An example of an attacker step that consists of multiple actions        is the recovering of a password enabling access to a given        network node based on an ability to remotely execute arbitrary        code in the given network node (e.g. remotely executing in the        given network node code that obtains a hash code of a password        enabling access to the given network node, and then recovering        the corresponding password from its hash code as in the previous        example). In this example, the attacker step is conditioned on a        prior achievement by the attacker of obtaining an ability to        remotely execute arbitrary code in the given network node.    -   As can be seen from the above examples, the breaking out of a        potential attack plan into attacker steps is somewhat arbitrary.        The second example above including a single attacker step        consisting of two actions could have been defined to include two        separate attacker steps, each including a single action—the        first attacker step consisting of remotely executing in the        given network node code that obtains the hash code, and the        second attacker step consisting of recovering the password from        its hash code.-   59. “remediation action” or just “remediation”—An action that    improves the security of a networked system by making one or more    attacker steps practically unavailable, more expensive, more    difficult, less efficient and/or less useful for attackers of the    networked system.    -   An example of a remediation action that makes only a single        attacker step practically unavailable to attackers is the        installing of a defensive measure applicable only to a single        network node (e.g. installing in a single network node a        software utility that locally requires fingerprints        identification on top of requiring a password in order to allow        access).    -   An example of a remediation action that makes multiple attacker        steps practically unavailable to attackers is the replacing of a        common algorithm or a common method used in multiple network        nodes of the networked system by an improved algorithm or method        (e.g. the global replacing of a simple password hash code        calculation algorithm by an improved password hash code        algorithm that uses salt in its calculation). In such case, each        given network node benefiting from the improved algorithm        corresponds to a different attacker step targeting the given        network node.    -   A remediation action that makes the one or more attacker steps        practically unavailable does not necessarily make the one or        more attacker steps completely unavailable to the attackers. If        an action makes the one or more attacker steps too costly for        the attackers to use (i.e. makes the cost of exploitation of the        one or more attacker steps so high that there is very low        probability that the attackers would use them), then the action        is considered to make the one or more attacker steps practically        unavailable to the attackers and therefore is a remediation        action.    -   Another example of a remediation action that does not make the        one or more attacker steps completely unavailable to the        attackers is an action of replacing an encryption algorithm        using a short key with a similar encryption algorithm using a        longer key. This may result in the deciphering of the encrypted        data by an attacker taking a much longer time than before. This        in turn makes the one or more attacker steps less efficient to        use, and therefore such action is considered to be a remediation        action.-   60. “sub-goal” or “achievement”—A result or ability obtained by an    attacker by successfully performing an attacker step against a    networked system, where the attacker was not in possession of the    result or ability before performing the attacker step.    -   For example, obtaining a password enabling access to a given        network node of the networked system is a possible sub-goal.        Such sub-goal may be obtained by an attacker by carrying out the        attacker step “recovering of a password to the given network        node from a known password hash code” or the attacker step        “exporting out of the networked system of a file containing all        credentials for the local network in which the given network        node resides”. As another example, exporting a specific file        (e.g. an Excel file containing a company's budget) out of the        networked system is a possible sub-goal. As still another        example, compromising a given network node so that the attacker        gets full control of it is a possible sub-goal. It should be        noted that every resource of a tested networked system        corresponds to a possible sub-goal of obtaining, controlling or        damaging that resource.    -   A special case of a sub-goal is a dummy sub-goal representing a        state in which an attacker has no possession of any result or        ability which is not available to all. A dummy sub-goal is        assumed to be achievable by any attacker even before carrying        out any attacker step.    -   In some cases, a sub-goal may be automatically obtained by an        attacker following the obtaining of another sub-goal, without        having to perform any additional attacker step. For example, the        sub-goal “compromise either node X or node Y” is automatically        obtained once the sub-goal “compromise node X” was obtained,        without having to perform any additional attacker step. In such        case it can be assumed that the sub-goal is obtained by a dummy        attacker step that does nothing.-   61. “path of attack of a networked system” or “a branch of attack of    a networked system”—An ordered sequence of one or more attacker    steps and one or more sub-goals in the networked system, that starts    with a starting sub-goal that is assumed to be achievable by    attackers and ends with a final sub-goal that is assumed to be a    goal of attackers when attacking the networked system.    -   The starting sub-goal may be a dummy sub-goal representing a        state in which an attacker has no possession of any result or        ability which is not available to all. Alternatively, the        starting sub-goal may be a non-dummy sub-goal representing a        state in which an attacker is in possession of a result or an        ability not available to all, which sub-goal is assumed to be        available to the attacker for the purpose of the present        testing.    -   A path of attack may be represented using multiple different        representation forms, including but not limited to various forms        of graphs and lists. In other words, different representation        forms of a path of attack may represent the same path of attack,        in which case they are equivalent to each other.    -   Typically, but not necessarily, a given path of attack satisfies        the following conditions: (A) each attacker step included in the        given path of attack is preceded by a sub-goal and followed by a        sub-goal in the given path of attack, (B) each sub-goal included        in the given path of attack, except for the starting sub-goal of        the given path of attack, is preceded by an attacker step in the        given path of attack, (C) each sub-goal included in the given        path of attack, except for the final sub-goal of the given path        of attack, is followed by an attacker step in the given path of        attack.-   62. “a true sub-goal of a path of attack”—A sub-goal included in the    path of attack that is not the starting sub-goal or the final    sub-goal of the path of attack.-   63. “an attacker step leading to a sub-goal”—An attacker step that,    if executed, achieves, or has a non-negligible probability of    achieving, the sub-goal for the attacker. The term “leading” in this    context should be understood as “directly leading”. In other words,    if attacker step A brings the attacker from sub-goal 1 to sub-goal 2    and attacker step B brings the attacker from sub-goal 2 to sub-goal    3, then attacker step A is leading to sub-goal 2 and attacker step B    is leading to sub-goal 3, but attacker step A is not leading to    sub-goal 3 (even though it is in a path of attack leading to    sub-goal 3).-   64. “importance of a sub-goal in a networked system”—A measure of    how important it is for the owner of the networked system to protect    the sub-goal against attack. For example, a sub-goal that is    included in a large number of paths of attack that are available to    attackers of the networked system is more important to protect than    a sub-goal included only in a single path of attack (assuming all    other factors associated with the two sub-goals are equal).    -   The importance of sub-goals may be represented by a numeric        score within a given range, typically (but not necessarily) with        a higher score indicating a sub-goal having a higher importance.        For example, the given range may be [0 . . . 10], with the        importance of a first sub-goal included in three paths of attack        being 8, and the importance of a second sub-goal included in a        single path of attack being 2. Alternatively, a lower score may        represent a lower importance sub-goal.-   65. “blocking an attacker step”, “blocking a vulnerability”,    “blocking a way for an attacker to compromise”—Making the attacker    step or the exploitation of the vulnerability or the way to    compromise (as the case may be) practically unavailable, more    expensive, more difficult, less efficient and/or less useful to    attackers. The blocking of the attacker step or the exploitation of    the vulnerability or the way to compromise is done by implementing    one or more remediation actions.-   66. “protecting a sub-goal”—Making the sub-goal practically    unachievable, more expensive to achieve, more difficult to achieve,    less efficient to achieve and/or less useful to attackers. The    protecting of the sub-goal is done by implementing one or more    remediation actions. Typically, the one or more remediation actions    applied for protecting a sub-goal are remediation actions that block    attacker steps leading to that sub-goal in at least one path of    attack in which it is included.-   67. “cost of exploitation of an attacker step”, “cost of    exploitation of a vulnerability”—A measure of how difficult or    expensive it is for an attacker to use the attacker step or to    exploit the vulnerability (as the case may be). For example, an    attacker step using the attacking method known as “ARP Spoofing” is    costlier for the attacker than an attacker step using a method of    attack taken from a publicly available exploit kit.    -   The cost of exploitation may be represented by a numeric score        within a given range, typically (but not necessarily) with a        higher score indicating a costlier attacker step. For example,        the given range may be [0 . . . 10], with the cost of        exploitation using ARP Spoofing being 7, and the cost of        exploitation using a method taken from a publicly available        exploit kit being 2. Alternatively, a lower score may represent        a costlier attacker step.-   68. “cost of remediation of an attacker step”, “cost of remediation    of a vulnerability”—A measure of how difficult or expensive it is    for the organization owning the networked system to which the    attacker step or the exploitation of the vulnerability is applied to    block the attacker step or the exploitation of the vulnerability (as    the case may be). For example, an attacker step that can be blocked    by simply installing a security patch for a software application    (e.g. Microsoft Word) is much less costly to block than an attacker    step that requires buying and installing a new router in order to    split an existing sub-network into two different sub-networks.    -   The cost of remediation may be represented by a numeric score        within a given range, typically (but not necessarily) with a        higher score representing a costlier attacker step. For example,        the given range may be [0 . . . 10], with the cost of a        remediation action requiring only installing a patch being 1,        and the cost of a remediation action requiring a new router        being 8. Alternatively, a lower score may represent a costlier        attacker step.-   69. “cost of remediation of a sub-goal”—A measure of how difficult    or expensive it is for the organization owning the networked system    in which the sub-goal is defined to protect the sub-goal against    being achieved by an attacker.    -   The cost of remediation of a sub-goal typically depends on the        costs of remediation of the attacker steps leading to it in the        paths of attack in which it is included.    -   For example, for a sub-goal included only in a single path of        attack, the cost of remediation of the sub-goal may be equal to        the cost of remediation of the single attacker step leading to        it in the single path of attack. For a sub-goal included in        multiple paths of attack, the cost of remediation of the        sub-goal may be equal to the sum of the costs of remediation of        all attacker steps leading to it in at least one path. If the        same attacker step leads to the sub-goal in multiple paths of        attack, the cost of remediation of that attacker step may be        taken into account only once.    -   The cost of remediation of a sub-goal may depend on factors        other than costs of remediation of attacker steps. For example,        the cost of remediation of a sub-goal may be the sum of the        multiplications of cost of remediation and probability of        success for all attacker steps leading to it in at least one        path.-   70. “probability of success of an attacker step”, “probability of    success of a vulnerability”—A measure of how probable is it that    execution of the attacker step or an attempt to exploit the    vulnerability (as the case may be) by the attacker will succeed in    achieving the sub-goal that the attacker step is intended to achieve    or will succeed in compromising the networked system, taking into    account currently available knowledge regarding the state of the    attacked networked system. For example, an attacker step that is    based on exploiting a known Windows 7 vulnerability may have high    probability of success when applied to a network node having the    original version of the OS installed, while having a low probability    of success when applied to a network node in which a certain    security patch had also been installed.    -   Typically, probabilities of success are expressed in percentages        in the range of 0% to 100%. Alternatively, the probabilities of        success may be represented by numeric values in the range of        zero to one, where zero corresponds to 0% and one corresponds to        100%. However, any other numerical scale may be used for        representing probabilities of success, provided that the scale        is a monotonically increasing or monotonically decreasing        function of how probable is it that the attacker step will        succeed in achieving its sub-goal.-   71. “a Boolean condition”—A statement that can have a value of    either true or false. If the statement is true, we say that the    Boolean condition is satisfied. If the statement is false, we say    that the Boolean condition is not satisfied.-   72. “subset/subgroup of a given set/group” or “sub-set/sub-group of    a given set/group”—A set/group that satisfies the condition that    that every member of it is also a member of the given set/group.    Unless otherwise stated, a subset/subgroup may be empty and contain    no members at all. Unless otherwise stated, a subset/subgroup of a    given set/group may contain all the members of the given set/group    and be equal to the given set/group.-   73. “proper subset/subgroup of a given set/group” or “proper    sub-set/sub-group of a given set/group”—A subset/subgroup of the    given set/group that is not equal to the given set/group. In other    words, there is at least one member of the given set/group that is    not a member of the subset/subgroup.-   74. “or”—A logical operator combining two Boolean input conditions    into a Boolean compound condition, such that the compound condition    is satisfied if and only if at least one of the two input conditions    is satisfied. In other words, if condition C=condition A or    condition B, then condition C is not satisfied when both condition A    and condition B are not satisfied, but is satisfied in each of the    following cases: (i) condition A is satisfied and condition B is not    satisfied, (ii) condition A is not satisfied and condition B is    satisfied, and (iii) both condition A and condition B are satisfied.-   75. “one of A and B”—If A and B are specific items, then “one of A    and B” is equivalent to “only A or only B, but not both”. For    example, “one of John and Mary” is equivalent to “only John or only    Mary, but not both John and Mary”. If A and B are categories, then    “one of A and B” is equivalent to “only one of A or only one of B,    but not both one of A and one of B”. For example, “one of a dog and    a cat” is equivalent to “only one dog or only one cat, but not both    one dog and one cat”. Similarly, if A and B are specific items, then    “at least one of A and B” is equivalent to “only A or only B, or    both A and B”. For example, “at least one of John and Mary” is    equivalent to “only John or only Mary, or both John and Mary”. If A    and B are categories, then “at least one of A and B” is equivalent    to “only at least one of A or only at least one of B, or both at    least one of A and at least one of B”. For example, “at least one of    a dog and a cat” is equivalent to “only at least one dog or only at    least one cat, or both at least one dog and at least one cat”.    -   Note that in “one of dogs and cats”, “dogs” and “cats” are not        categories but specific groups (i.e. specific items). Therefore,        “one of dogs and cats” is equivalent to “only dogs or only cats,        but not both dogs and cats”. Similarly, “at least one of dogs        and cats” is equivalent to “only dogs or only cats, or both dogs        and cats”.    -   If A, B and C are specific items, then “one of A, B and C” is        equivalent to “only A or only B or only C, but not a combination        of two or three members of the group consisting of: A, B and C”,        and “at least one of A, B and C” is equivalent to “only A or        only B or only C, or any combination of two or three members of        the group consisting of: A, B and C”.    -   If A, B and C are categories, then “one of A, B and C” is        equivalent to “only one of A or only one of B or only one of C,        but not a combination of two or three members of the group        consisting of: one of A, one of B and one of C”, and “at least        one of A, B and C” is equivalent to “only at least one of A or        only at least one of B or only at least one of C, or any        combination of two or three members of the group consisting of:        one of A, one of B and one of C”.    -   If the list following the “one of” or the “at least one of”        contains more than three members, then the previous definitions        are again applicable, with the appropriate modifications that        extrapolate the above logic.    -   Note that “one or more of” is equivalent to “at least one of”,        and the two terms are synonyms.

It will be appreciated that certain features of the invention, whichare, for clarity, described in the context of separate embodiments, mayalso be provided in combination in a single embodiment. Conversely,various features of the invention, which are, for brevity, described inthe context of a single embodiment, may also be provided separately, inany other embodiment, or in any suitable combination including only asub-group of those features.

Although the invention has been described in conjunction with specificembodiments thereof, it is evident that many alternatives, modificationsand variations will be apparent to those skilled in the art.Accordingly, it is intended to embrace all such alternatives,modifications and variations that fall within the spirit and broad scopeof the appended claims. All publications, patents and patentapplications mentioned in this specification are herein incorporated intheir entirety by reference into the specification, to the same extentas if each individual publication, patent or patent application wasspecifically and individually indicated to be incorporated herein byreference. In addition, citation or identification of any reference inthis application shall not be construed as an admission that suchreference is available as prior art to the present invention.

The invention claimed is:
 1. A method for providing, by a penetrationtesting system, a recommendation for improving the security of anetworked system against attackers, the method comprising: a. carryingout one or more penetration tests of the networked system by thepenetration testing system; b. based on results of the one or morepenetration tests of the networked system, determining multiple paths ofattack available to the attackers, each path of attack of the determinedmultiple paths of attack being an ordered sequence of one or moreattacker steps and one or more sub-goals; c. assigning a calculatedimportance score to each of multiple sub-goals, wherein (i) eachsub-goal of the multiple sub-goals is included in at least one of thedetermined multiple paths of attack, and (ii) for at least one givensub-goal of the multiple sub-goals, the importance score assigned to thegiven sub-goal is based on a number of paths of attack of the determinedmultiple paths of attack which include the given sub-goal; d. selectingone sub-goal included in at least one of the determined multiple pathsof attack, the selecting of the one sub-goal being based on theimportance score assigned to at least one of the multiple sub-goals; ande. providing a recommendation to protect the selected one sub-goal, theproviding of the recommendation comprising at least one operationselected from the group consisting of: i. causing a display device todisplay information about the recommendation; ii. recording theinformation about the recommendation in a file; and iii. electronicallytransmitting the information about the recommendation.
 2. The method ofclaim 1, wherein, for each given sub-goal of the multiple sub-goals, theimportance score assigned to the given sub-goal is based on the numberof paths of attack of the determined multiple paths of attack whichinclude the given sub-goal.
 3. The method of claim 1, wherein, for atleast one given sub-goal of the multiple sub-goals, the importance scoreassigned to the given sub-goal is based only on the number of paths ofattack of the determined multiple paths of attack which include thegiven sub-goal.
 4. The method of claim 1, wherein, for each givensub-goal of the multiple sub-goals, the importance score assigned to thegiven sub-goal is based only on the number of paths of attack of thedetermined multiple paths of attack which include the given sub-goal. 5.The method of claim 1, wherein, for at least one given sub-goal of themultiple sub-goals, the importance score assigned to the given sub-goalis equal to the number of paths of attack of the determined multiplepaths of attack which include the given sub-goal.
 6. The method of claim1, wherein the selecting of the one sub-goal included in at least one ofthe determined multiple paths of attack comprises selecting one sub-goalwhose assigned importance score meets a predefined criterion.
 7. Themethod of claim 1, wherein, for at least one given sub-goal of themultiple sub-goals, the importance score assigned to the given sub-goalis based on (i) the number of paths of attack of the determined multiplepaths of attack which include the given sub-goal, and (ii) at least onemember selected from the group consisting of: a cost of remediation ofthe given sub-goal, a cost of exploitation of an attacker step leadingto the given sub-goal, a cost of remediation of an attacker step leadingto the given sub-goal, and a probability of success of an attacker stepleading to the given sub-goal.
 8. A method for providing, by apenetration testing system, a recommendation for improving the securityof a networked system against attackers, the method comprising: a.initializing a list of sub-goals that should be protected to be an emptylist; b. obtaining a halting condition, the halting condition includinga Boolean condition applied to the list of sub-goals; c. carrying outone or more penetration tests of the networked system by the penetrationtesting system; d. based on results of the one or more penetration testsof the networked system, determining multiple paths of attack availableto the attackers, each path of attack of the determined multiple pathsof attack being an ordered sequence of one or more attacker steps andone or more sub-goals; e. initializing a group of relevant paths ofattack to consist of the determined multiple paths of attack; f.assigning a calculated importance score to each of one or moresub-goals, wherein (i) each sub-goal of the one or more sub-goals isincluded in at least one path of attack included in the group ofrelevant paths of attack, and (ii) for at least one given sub-goal ofthe one or more sub-goals, the importance score assigned to the givensub-goal is based on a number of paths of attack in the group ofrelevant paths of attack which include the given sub-goal; g. selectingone sub-goal included in at least one member of the group of relevantpaths of attack and adding the one sub-goal to the list of sub-goals,the selecting of the one sub-goal being based on the importance scoresassigned to at least one of the one or more sub-goals; h. modifying thegroup of relevant paths of attack by removing from it every path ofattack that includes the one sub-goal; i. evaluating the haltingcondition for the list of sub-goals; j. in response to determining that(i) the halting condition is not satisfied, and (ii) the group ofrelevant paths of attack is not empty, repeating steps f to j; and k. inresponse to determining that (i) the halting condition is satisfied, or(ii) the group of relevant paths of attack is empty, providing arecommendation to protect one or more sub-goals from the list ofsub-goals, the providing of the recommendation comprising at least oneoperation selected from the group consisting of: i. causing a displaydevice to display information about the recommendation; ii. recordingthe information about the recommendation in a file; and iii.electronically transmitting the information about the recommendation. 9.The method of claim 8, wherein, for each given sub-goal of the one ormore sub-goals, the importance score assigned to the given sub-goal isbased on the number of paths of attack in the group of relevant paths ofattack which include the given sub-goal.
 10. The method of claim 8,wherein, for at least one given sub-goal of the one or more sub-goals,the importance score assigned to the given sub-goal is based only on thenumber of paths of attack in the group of relevant paths of attack whichinclude the given sub-goal.
 11. The method of claim 8, wherein, for eachgiven sub-goal of the one or more sub-goals, the importance scoreassigned to the given sub-goal is based only on the number of paths ofattack in the group of relevant paths of attack which include the givensub-goal.
 12. The method of claim 8, wherein, for at least one givensub-goal of the one or more sub-goals, the importance score assigned tothe given sub-goal is equal to the number of paths of attack in thegroup of relevant paths of attack which include the given sub-goal. 13.The method of claim 8, wherein the selecting of the one sub-goalincluded in at least one member of the group of relevant paths of attackcomprises selecting one sub-goal whose assigned importance score meets apredefined criterion.
 14. The method of claim 8, wherein, for at leastone given sub-goal of the one or more sub-goals, the importance scoreassigned to the given sub-goal is based on (i) the number of paths ofattack in the group of relevant paths of attack which include the givensub-goal, and (ii) at least one member selected from the groupconsisting of: a cost of remediation of the given sub-goal, a cost ofexploitation of an attacker step leading to the given sub-goal, a costof remediation of an attacker step leading to the given sub-goal, and aprobability of success of an attacker step leading to the givensub-goal.
 15. The method of claim 8, wherein (i) the list of sub-goalsis an ordered list of sub-goals, (ii) the adding of the one sub-goal tothe list of sub-goals includes adding the one sub-goal at an end of theordered list of sub-goals such that the one sub-goal becomes a lastmember of the ordered list of sub-goals, and (iii) the providing arecommendation to protect one or more sub-goals from the list ofsub-goals includes providing a recommendation to protect the one or moresub-goals from the list of sub-goals according to an order of the one ormore sub-goals in the ordered list of sub-goals.
 16. The method of claim8, wherein the halting condition is true if and only if the list ofsub-goals consists of one sub-goal.
 17. The method of claim 8, whereinthe halting condition is true if and only if the list of sub-goalsconsists of a pre-determined number of sub-goals.
 18. The method ofclaim 8, wherein the halting condition is true if and only if a sum ofremediation costs of all members of the list of sub-goals satisfies asecond Boolean condition.
 19. A system for providing a recommendationfor improving the security of a networked system against attackers, thesystem comprising: a. a penetration-testing-campaign module including:i. one or more penetration-testing-campaign hardware processors; and ii.a penetration-testing-campaign non-transitory computer readable storagemedium for instructions execution by the one or morepenetration-testing-campaign hardware processors, thepenetration-testing-campaign non-transitory computer readable storagemedium having stored instructions to carry out one or more penetrationtests of the networked system; b. a sub-goal-selection module including:i. one or more sub-goal-selection hardware processors; and ii. asub-goal-selection non-transitory computer readable storage medium forinstructions execution by the one or more sub-goal-selection hardwareprocessors, the sub-goal-selection non-transitory computer readablestorage medium having stored: 1) instructions to receive, from thepenetration-testing-campaign module, results of the one or morepenetration tests of the networked system; 2) instructions to determine,based on said received results, multiple paths of attack available tothe attackers, each path of attack of the determined multiple paths ofattack being an ordered sequence of one or more attacker steps and oneor more sub-goals; 3) instructions to assign a calculated importancescore to each of multiple sub-goals, wherein (i) each sub-goal of themultiple sub-goals is included in at least one of the determinedmultiple paths of attack, and (ii) for at least one given sub-goal ofthe multiple sub-goals, the importance score assigned to the givensub-goal is based on a number of paths of attack of the determinedmultiple paths of attack which include the given sub-goal; and 4)instructions to select one sub-goal included in at least one of thedetermined multiple paths of attack, the selecting of the one sub-goalbeing based on the importance score assigned to at least one of themultiple sub-goals; and c. a reporting module including: i. one or morereporting hardware processors; and ii. a reporting non-transitorycomputer readable storage medium for instructions execution by the oneor more reporting hardware processors, the reporting non-transitorycomputer readable storage medium having stored: 1) instructions toreceive, from the sub-goal-selection module, an identification of theselected one sub-goal; and 2) instructions to provide a recommendationto protect the selected one sub-goal, the instructions to provide therecommendation including at least one member selected from the groupconsisting of: I. instructions to cause a display device to displayinformation about the recommendation; II. instructions to record theinformation about the recommendation in a file; and III. instructions toelectronically transmit the information about the recommendation.
 20. Asystem for providing a recommendation for improving the security of anetworked system against attackers, the system comprising: a. apenetration-testing-campaign module including: i. one or morepenetration-testing-campaign hardware processors; and ii. apenetration-testing-campaign non-transitory computer readable storagemedium for instructions execution by the one or morepenetration-testing-campaign hardware processors, thepenetration-testing-campaign non-transitory computer readable storagemedium having stored instructions to carry out one or more penetrationtests of the networked system; b. a sub-goals-selection moduleincluding: i. one or more sub-goals-selection hardware processors; andii. a sub-goals-selection non-transitory computer readable storagemedium for instructions execution by the one or more sub-goals-selectionhardware processors, the sub-goals-selection non-transitory computerreadable storage medium having stored: 1) first instructions toinitialize a list of sub-goals that should be protected to be an emptylist; 2) second instructions to obtain a halting condition, the haltingcondition including a Boolean condition applied to the list ofsub-goals; 3) third instructions to receive, from thepenetration-testing-campaign module, results of the one or morepenetration tests of the networked system; 4) fourth instructions todetermine, based on said received results of the one or more tests ofthe networked system, multiple paths of attack available to theattackers, each path of attack of the determined multiple paths ofattack being an ordered sequence of one or more attacker steps and oneor more sub-goals; 5) fifth instructions to initialize a group ofrelevant paths of attack to consist of the determined multiple paths ofattack; 6) sixth instructions to assign a calculated importance score toeach of one or more sub-goals, wherein (i) each sub-goal of the one ormore sub-goals is included in at least one path of attack included inthe group of relevant paths of attack, and (ii) for at least one givensub-goal of the one or more sub-goals, the importance score assigned tothe given sub-goal is based on a number of paths of attack in the groupof relevant paths of attack which include the given sub-goal; 7) seventhinstructions to select one sub-goal included in at least one member ofthe group of relevant paths of attack and to add the one sub-goal to thelist of sub-goals, the selecting of the one sub-goal being based on theimportance scores assigned to at least one of the one or more sub-goals;8) eighth instructions to modify the group of relevant paths of attackby removing from it every path of attack that includes the one sub-goal;9) ninth instructions to evaluate the halting condition for the list ofsub-goals; 10) tenth instructions, to be carried out in response todetermining that (i) the halting condition is not satisfied, and (ii)the group of relevant paths of attack is not empty, to repeat the sixthinstructions to the tenth instructions; and 11) eleventh instructions,to be carried out in response to determining that (i) the haltingcondition is satisfied, or (ii) the group of relevant paths of attack isempty, to select one or more sub-goals from the list of sub-goals; andc. a reporting module including: i. one or more reporting hardwareprocessors; and ii. a reporting non-transitory computer readable storagemedium for instructions execution by the one or more reporting hardwareprocessors, the reporting non-transitory computer readable storagemedium having stored: 1) instructions to receive, from thesub-goals-selection module, an identification of the one or moreselected sub-goals; and 2) instructions to provide a recommendation toprotect the one or more selected sub-goals, the instructions to providethe recommendation including at least one member selected from the groupconsisting of: I. instructions to cause a display device to displayinformation about the recommendation; II. instructions to record theinformation about the recommendation in a file; and III. instructions toelectronically transmit the information about the recommendation.